bpfis errors in problem report

[MikeM11]

I have a Windows 2003 (x86) Standard SP2 Client in a MS-Windows policy that is completing successfully as far as the Activity Monitor is concerned but faor the last couple of weeks we have been seeing entries as below in the Problem Report:

Error from client clientname.domainname.co.uk: ERR - Send bpfis state to masterserver.domainname.co.uk failed. status = 25

Error from client clientname.domainname.co.uk: ERR - Get bpfis state from masterserver.domainname.co.uk failed. status = 25

Error from client clientname.domainname.co.uk: ERR - Delete bpfis state from masterserver.domainname.co.uk failed. status = 25

The Microsoft VSS rollup pack has since been applied to the server which is know to resolve VSS problems in 2003 but the issue remains.

My environment is Windows 2008 R2 (x64) Enterprise for Master/Media running NetBackup 7.1.0.4

Has anyone else come across this issue before or have any idea what may be causing it?

Many thanks, Mike

[V4]

What are VSS writers state.

Pls post bpfis  logs (verbose 5)

[Yogesh9881]

kindly post o/p of below command

run on client..

c:\>vssadmin list writers

c:\>vssadmin list providers

<install path>\veritas\netbackup\bin>bpclntcmd -pn

run on master ....

<install path>\veritas\netbackup\bin\admincmd>bptestbpcd -client <client_name> -verbose -debug

[MikeM11]

VSS Writers

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001 Microsoft Corp.

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {ce85a722-f6d1-4593-afa4-33eb238bf138}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {a2ec1f06-0504-4acf-a0df-d394784e1f34}
   State: [1] Stable
   Last error: No error

Writer name: 'MSDEWriter'
   Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}
   Writer Instance Id: {252626a3-6780-4245-906a-5fccda972045}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {803db1fe-6491-4a74-b26b-e913e273822c}
   State: [1] Stable
   Last error: No error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {db59fcec-c41d-4080-ad59-9e0a42f14750}
   State: [1] Stable
   Last error: No error

Writer name: 'Event Log Writer'
   Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}
   Writer Instance Id: {074470a4-43e4-43e5-90ea-8956ed360b87}
   State: [1] Stable
   Last error: No error

Writer name: 'IIS Metabase Writer'
   Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
   Writer Instance Id: {30e5f3f0-302c-48b6-a658-4614fa326edd}
   State: [1] Stable
   Last error: No error

VSS Providers:

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001 Microsoft Corp.

Provider name: 'Microsoft Software Shadow Copy provider 1.0'
   Provider type: System
   Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
   Version: 1.0.0.7

I ran bpclntcmd -pn on the Client and bptestbpcd on the Master

1 1 1
masterserver_ip:49894 -> client_ip:1556
masterserver_ip:49895 -> client_ip:1556
PEER_NAME = masterserver.domain.co.uk
HOST_NAME = client.domain.co.uk
CLIENT_NAME = client.domain.co.uk
VERSION = 0x07100000
PLATFORM = win_x86
PATCH_VERSION = 7.1.0.3
SERVER_PATCH_VERSION = 7.1.0.3
MASTER_SERVER = masterserver.domain.co.uk
EMM_SERVER = masterserver.domain.co.uk
NB_MACHINE_TYPE = CLIENT
masterserver_ip:49896 -> client_ip:1556

Thanks, Mike

[MikeM11]

I have just noted this extract from the client bpfis log:

13:52:28.529 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:52:29.466 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:52:59.467 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:53:00.749 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:53:10.468 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:53:40.469 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:53:42.469 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:53:51.375 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:54:21.376 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:54:25.376 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:54:32.298 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:55:02.299 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:55:10.299 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:55:13.331 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:55:34.315 [1204.2348] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1376: 0: getsockopt SO_ERROR returned: 10060 0x0000274c
13:55:34.315 [1204.2348] <2> connect_to_service: ../../libvlibs/vnet_connect.c.382: 0: vnet_async_connect() failed: 18 0x00000012
13:55:34.315 [1204.2348] <2> vnet_connect_to_service: ../../libvlibs/vnet_connect.c.174: 0: connect_to_service() failed: 18 0x00000012
13:55:34.315 [1204.2348] <2> bprd_connect: vnet_connect_to_service(masterserver.domain.co.uk) failed: 18
13:55:34.315 [1204.2348] <2> bprd_bpfis_state_xfer_to_client_rqst: bprd_connect(masterserver.domain.co.uk) failed, cannot connect on socket (25)
13:55:34.315 [1204.2348] <16> bpfis: ERR - Get bpfis state from masterserver.domain.co.uk failed. status = 25

[Mark_Solutions]

It may be that you are getting a timeout during the snapshot and file preparation period

Increase the client read timeout on your Media Servers (1800 usually works well) and see if this clears it up.

Also, depending on the version of the client you have installed make sure the client is added to the Master Server Host Properties - Client Attributes section and on the Windows Open File Backup tab set it to use VSS, Individual Drive Snapshots and Disable Snapshot and Continue - just stops it from using VSP if it has an older client installed.

Hope this helps

[MikeM11]

We already have the timeout set to 1800 and have VSS configured as stated.  The Client version is 7.1.0.3 as we have not got round to deploying MP4 for Clients due to issues we have been having since upgrading the Master/Media servers to 7.1.0.4.

Mike

[Mark_Solutions]

Is it just that one client?

Is it a busy server in terms of tcpip?

2003 has a few tcpip issues and maybe it just runs out of ports or connections get stuck in a time_wait state

run a netstat -a on it to see how busy it is and then you could add the following keys to its registry (need a reboot):

HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\

DWORD – TcpTimedWaitDelay  - Decimal Value of 30

DWORD – MaxUserPort – Decimal Value 65534

This is also handy on your Master and Media Servers to increase network capabilities but bear in mind that Windows 2008 does not use this in same way as 2003 so instead of the MaxUserPort setting do the following from an administrative command line

netsh int ipv4 set dynamicport tcp start=10000 num=50000

Hope this helps

[Yogesh9881]

just for testing .....

On client host  -- are you able to take system state backup by using NTBACKUP ??

[MikeM11 · Accepted solution]

This has now been resolved and it turned out to be down to inactive persistent routes on the Windows server.

Due to various VLAN restrictions within my enterprise and our backup networks not being routed we have to use persistent routes in some cases.  These had been applied to the Client but had dropped out of the active routes table.  Reapplying the routes resolved the problem.

Many thanks for your assistance anyway. Mike