The status code 58 Can’t connect to client

[ea021e]

Hi, I am having an issue with 6 new Windows 2008 servers, the network communication is ok (ping and trace), dns resolves ok name and ip in client, media and master and bpclntcmd gives the correct response (-pn -hn -ip and -self). The issue is that bptestbpcd works ok from master server but from media fails with the following output:

./bptestbpcd -verbose -debug -client casndg1cqhubw06-nebr.sddc.sbc.com
01:55:02.145 [24252] <2> bptestbpcd: VERBOSE = 1
01:55:02.219 [24252] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
01:55:02.219 [24252] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpdbm
01:55:02.322 [24252] <2> logconnections: BPDBM CONNECT FROM 135.161.151.97.38022 TO 135.161.150.167.13724
01:58:47.304 [24252] <2> vnet_async_connect: vnet_vnetd.c.4237: getsockopt SO_ERROR returned: 145 0x00000091
01:58:57.304 [24252] <2> vnet_async_connect: vnet_vnetd.c.4237: getsockopt SO_ERROR returned: 145 0x00000091
02:00:04.312 [24252] <2> vnet_async_connect: vnet_vnetd.c.3812: ran out of time before connect: 302 0x0000012e
02:00:04.312 [24252] <2> vnet_async_connect: vnet_vnetd.c.3812: ran out of time before connect: 302 0x0000012e
02:00:04.313 [24252] <2> vnet_connect_to_service_or_vnetd: vnet_vnetd.c.3571: vnet_async_connect failed: 18 0x00000012
02:00:04.313 [24252] <2> nb_connect_to_vnetd_or_legacy: comm.c.2030: vnet_connect_to_vnetd_or_service failed: 18
02:00:04.313 [24252] <2> local_bpcr_connect: nb_connect_to_vnetd_service(casndg1cqhubw06-nebr.sddc.sbc.com) failed: 25
02:00:04.313 [24252] <2> local_bpcr_connect: Can't connect to client casndg1cqhubw06-nebr.sddc.sbc.com
02:00:04.313 [24252] <2> ConnectToBPCD: bpcd_connect_and_verify(casndg1cqhubw06-nebr.sddc.sbc.com, casndg1cqhubw06-nebr.sddc.sbc.com) failed: 25
<16>bptestbpcd main: Function ConnectToBPCD(casndg1cqhubw06-nebr.sddc.sbc.com) failed: 25
02:00:04.314 [24252] <16> bptestbpcd main: Function ConnectToBPCD(casndg1cqhubw06-nebr.sddc.sbc.com) failed: 25
<2>bptestbpcd: cannot connect on socket
02:00:04.314 [24252] <2> bptestbpcd: cannot connect on socket
<2>bptestbpcd: EXIT status = 25
02:00:04.314 [24252] <2> bptestbpcd: EXIT status = 25
cannot connect on socket
 

I tried to bounce the service in the client and rebooted also, I have enable verbose 5 logs for bpcd and vnetd. Also a very similar client is working ok with the master/media servers. Found this in the vnetd logs in the client

01:27:59.686 [2796.5276] <2> logparams: D:\PROGRA~1\Veritas\NETBAC~1\bin\VNETD.EXE
01:27:59.686 [2796.5276] <2> ProcessRequests: .\vnetd.c.288: msg: VNETD ACCEPT FROM 130.1.5.23.64316 TO 10.112.136.91.13724 fd = 616
01:29:54.097 [2796.5276] <2> vnet_sock_ready: ..\libvlibs\vnet.c.507: max_time: 10 0x0000000a
01:29:54.097 [2796.5276] <2> vnet_sock_ready: ..\libvlibs\vnet.c.508: sock: 616 0x00000268
01:29:54.097 [2796.5276] <2> vnet_sock_ready: ..\libvlibs\vnet.c.509: Function failed: 11 0x0000000b
01:29:54.097 [2796.5276] <2> vnet_pop_byte: ..\libvlibs\vnet.c.173: Function failed: 11 0x0000000b
01:29:54.097 [2796.5276] <2> vnet_pop_string: ..\libvlibs\vnet.c.268: Function failed: 11 0x0000000b
01:29:54.097 [2796.5276] <2> vnet_pop_signed: ..\libvlibs\vnet.c.312: Function failed: 11 0x0000000b
01:29:54.097 [2796.5276] <2> vnet_version_accept: .\vnetd.c.1110: Function failed: 11 0x0000000b
01:29:54.097 [2796.5276] <2> ProcessRequests: .\vnetd.c.292: version_accept failed: 11 0x0000000b
01:29:54.097 [2796.5276] <2> ListenForConnection: .\vnetd.c.594: Function failed: 11 0x0000000b
01:29:54.097 [2796.5276] <16> main: Terminating with status 11

130.1.5.23 ip of the medio and 10.112.136.91 ip of the client.

I believe that something is blocking the vnetd communication but cant find the evidence to send it to the system admin of the server, would appreciate any help.

Master server version 6.5.6 same as media and client.

Thanks

[Marianne]

Please ensure bpcd log exists on the client, do bptestbpcd again from media server, then post client's bpcd log file. 

If file is more than a couple of lines, please rename file to bpcd.txt and post as attachment.

[Nicolai]

Does the Windows 2008 firewall allow access to the media server also ?

[ea021e]

No firewall in the server and telnet gives the correct respomse to 13724 and 13782, netstat shows both port open and listening.

The bptestbpcd from the media did not generate logs in the bpcd, I think is because it is not passing the handshake with the server.

[Marianne]

If telnet works and both ports are listening, then there is no reason why log file will not be generated....
Unless there is a problem with actual NBU binaries.

Try to remove patch and client software, then reinstall and patch.

[ea021e]

I have already done that, re-install everything, reboot and tried again. The weird part is that the master server has good communication and if I use it as media the backup runs ok. But the media servers that has issues is backing up other similar clients without problems.

[matt077]

does a netstat -a on client shows its listening on port 13724 and 13782?

Even though Firewall has been turned off from control panel > Windows Firewall, The Firewall Service must also be stopped for a successful connection.

[ea021e]

Yes netstat -a shows the ports listening and I have stopped and disabled the windows firewall service.

[matt077]

What happens if you try this from the server :

telnet <ip address of client> 13782

[Marianne]

Forget what I said about NBU binaries - connection from Master is working.

If we look at bptestbpcd output, we see that the media server and client are on different networks. 

You say routing between media server and client network is fine?
media can server ping the client?
client can ping the media server?
telnet to client bpcd and vnetd ports from the media server are both working fine?

Are you using hostname when performing all of above tests or IP address?

If you don't mind, please show us output of all of these tests done on the media server?

Please also test telnet from the client to media server's vnetd port.

Also do bptestbpcd from the master - confirm that bpcd log on client is written to.

[ea021e]

Network communication is ok in both ways, below some of the tests

ea021e@ssprd201: ping casndg1cqhubw06-nebr.sddc.sbc.com
casndg1cqhubw06-nebr.sddc.sbc.com is alive
ea021e@ssprd201: telnet casndg1cqhubw06-nebr.sddc.sbc.com 13782
Trying 10.112.136.91...
Connected to casndg1cqhubw06-nebr.sddc.sbc.com.
Escape character is '^]'.
^CConnection to casndg1cqhubw06-nebr.sddc.sbc.com closed by foreign host.
ea021e@ssprd201: telnet casndg1cqhubw06-nebr.sddc.sbc.com 13724
Trying 10.112.136.91...
Connected to casndg1cqhubw06-nebr.sddc.sbc.com.
Escape character is '^]'.
^CConnection to casndg1cqhubw06-nebr.sddc.sbc.com closed by foreign host.

I also tested using the ip and it works ok.

Telnet from client to media vnetd works ok (as the usual blank screen) also the telnet to loopback ip using bpcd and vnetd

bptestbpcd from master to client done and yes it wrote a log with the success.

[ea021e · Accepted solution]

Finally the issue was in the client, it has multiple nic so we had to run this command to enacle the communication using the path we required.

netsh interface ipv4 set interface [InterfaceNameOrIndex] weakhostsend=enable

netsh interface ipv4 set interface [InterfaceNameOrIndex] weakhostreceive=enable

http://www.symantec.com/docs/TECH63457 - Backup and restore operation for Windows 2008 clients fail with status 25 when the host is multi-homed

Anyway, thanks for all your time and atention.