Error Code 25 - Cannot connect on socket

[BR549]

This issue just started yesterday after apply the Monthly Microst Patches for Mar 2017 to the client server (2012 Std) and the Master\Media Server (2008 R2 SP1) then rebooting the servers.  These servers were warm rebooted and not shut down then started.  

- From the Veritas Master\Media server (version 7.5.0.7) I can successfully telnet to the client by hostname and IP Address on port 13782.

- From the client I can successfully telnet to the Master\Media server by hostname and IP Address on port 13720.

- I have restarted the NetBacup services on the client via the services.msc and utilizing bpdown and bpup commands.

-Both the Master\Media server can ping each other by hostname and IP Address.  bpclntcmd -pn & bpclntcmd -ip both appear to return the appropriate responses.

- On the Master\Media server I have removed the client from the policies then recreated the client for the policies to preclude a possible bad profile as the cause of the issue.

I have attached diagnostics that I've run from the servers to this posting.  This is a production server and the IP Addresses and servernames in the attachment have been modified to have place holders in the 1st 3 octets for privacy concerns.

I am not sure what to try next as so far everything that I've read and tried has not resolved the issue.  I am not ruling out the need to perform a cold reboot of the client server since the Master\Media server does not have any issues with the other servers who received the same changes as the client server experiencing the issue.

[Tape_Archived]

Please review your registery setting on the clients, it should have the name of the Master and media servers.

HKLM\SOFTWARE\Veritas\NetBackup\CurrentVersion\Config\Server

[GeForce123]

What is this? It looks like you've done 0 testing between master/media/client and whatever the Y.Y.Y.52 is

09:19:53.608 [10172.10280] <2> logconnections: BPCD ACCEPT FROM Y.Y.Y.52.52737 TO Y.Y.Y.51.13782 fd = 584

09:19:53.608 [10172.10280] <2> process_requests: setup_sockopts complete

09:19:53.652 [9260.9908] <2> logconnections: BPCD ACCEPT FROM Y.Y.Y.52.52738 TO Y.Y.Y.51.13782 fd = 584

09:19:58.271 [10172.10280] <8> retry_getnameinfo: [vnet_addrinfo.c:1151] getnameinfo() failed 11001 0x2af9

09:19:58.271 [10172.10280] <8> vnet_cached_getnameinfo: [vnet_addrinfo.c:1900] retry_getnameinfo() failed RV=11001 IPSTR=Y.Y.Y.52 PORT=52737

09:19:58.271 [10172.10280] <2> bpcd peer_hostname: vnet_cached_getnameinfo() failed: 6 11001

09:19:58.271 [10172.10280] <2> bpcd peer_hostname: Connection from host Y.Y.Y.52 (Y.Y.Y.52) port 52737

09:19:58.271 [10172.10280] <2> bpcd valid_server: comparing gxxxx102 and Y.Y.Y.52

09:19:58.272 [10172.10280] <2> bpcd valid_server: comparing X.X.X.71 and Y.Y.Y.52

09:19:58.272 [10172.10280] <4> bpcd valid_server: Y.Y.Y.52 is not a master server

09:19:58.272 [10172.10280] <16> bpcd valid_server: Y.Y.Y.52 is not a media server either

09:19:58.571 [10172.10280] <16> process_requests: read failed: An existing connection was forcibly closed by the remote host.

You do testing from the client which is X.X.X.78 and your master server which is X.X.X.71

[BR549]

I checked the registry setting,HKLM\SOFTWARE\Veritas\NetBackup\CurrentVersion\Config\Server, on the client server per your suggestion and it shows the Master\Media server hostname in the registry.

The environment has been up and operating for years and the client has been successfully backing up to the Master\Media server through the early morning hours on 26 Mar 2017.  The issue first appeared after the client was rebooted to apply March 2017 Microsoft Patches to the server when the user backup jobs kicked off.

[BR549]

As I indicated in my original post for privacy concerns I replaced the 1st 3 octets of the IP Addresses with placeholders.  I used X.X.X for one subnet and Y.Y.Y for a different subnet that appeared in the results of the bptestbpcd -verbose -debug -client fxxxxxxx4 command.  Why Y.Y.Y.52 showed up in the test from the Master\Media server to the client I do not know since that IP Address has never been a Master or a Media server in the environment.

[BR549 · Accepted solution]

I found the solution to the issue and the  HTML link is given below.  I have modified the text slightly to put a (-) sign in front of the client in the bpclient commands as it would not work without it.  Even thought NBU Admin console showed the client hostname I had to ADD the client hostname with the communication parameters as shown below then to be sure I did the UPDATE command.  Once they were done I performed the 

https://www.veritas.com/support/en_US/article.000108081

Solution

1. Test the connection using the following command :  Windows :

<Install_Path>\VERITAS\NetBackup\bin\admincmd\bptestbpcd -client <CLIENTNAME> -connect_options 1 0 2​
 

Note: If the bptestbpcd fails to connect, review the bptestbpcd output using verbose switch to troubleshoot further.

1. If this works, update the settings of the client on the master server using the following Windows :

<Install_Path>\VERITAS\NetBackup\bin\admincmd\

a.  bpclient -update -client <CLIENTNAME> -connect_options 1 0 2

b.  If the client is not listed in the client list need to add it with

       bpclient -add -client <CLIENTNAME> -connect_options 1 0 2

Test the connection again using the following command :
 
bptestbpcd -client <CLIENTNAME>
 

1. Test the backup job.

Actual process utilized

C:\Program Files\Veritas\NetBackup\bin\admincmd>bpclient -add -client fxxxxxxxx4 -connect_options 1 0 2

 C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -client fxxxxxxxx4

1 0 2

X.X.X.71:61813 -> X.X.X.78:13782

X.X.X.71:1276 <- X.X.X.78:50150

X.X.X.71:2467 <- X.X.X.78:50151

 C:\Program Files\Veritas\NetBackup\bin\admincmd>bpclient -update -client fxxxxxxxx4 -connect_options 1 0 2

 C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -client fxxxxxxxx4

1 0 2

X.X.X.71:62228 -> X.X.X.78:13782

X.X.X.71:1283 <- X.X.X.78:50487

X.X.X.71:2859 <- X.X.X.78:50488

C:\Program Files\Veritas\NetBackup\bin>bptestbpcd -verbose -debug -client fxxxxxxxx4

14:07:06.990 [11992.17684] <2> bptestbpcd: VERBOSE = 0

14:07:06.990 [11992.17684] <2> read_client: dname=., offline=0, online_at=0 offline_at=0

14:07:06.990 [11992.17684] <2> read_client: dname=.., offline=0, online_at=0 offline_at=0

14:07:07.005 [11992.17684] <2> read_client: dname=CO_0, offline=0, online_at=0 offline_at=0

14:07:07.005 [11992.17684] <2> read_client: dname=host_info, offline=0, online_at=0 offline_at=0

14:07:07.005 [11992.17684] <2> read_client: dname=OA_0, offline=0, online_at=0 offline_at=0

14:07:07.005 [11992.17684] <2> db_freeEXDB_INFO: ?

14:07:07.021 [11992.17684] <2> logconnections: BPCD CONNECT FROM X.X.X.71.62810 TO X.X.X.78.13782 fd = 940

1 0 2

X.X.X.71:62810 -> X.X.X.78:13782

X.X.X.71:3045 <- X.X.X.78:50627

14:07:07.286 [11992.17684] <2> bpcr_get_peername_rqst: Server peername length = 8

14:07:07.505 [11992.17684] <2> bpcr_get_hostname_rqst: Server hostname length = 10

14:07:07.723 [11992.17684] <2> bpcr_get_clientname_rqst: Server clientname length = 10

14:07:07.926 [11992.17684] <2> bpcr_get_version_rqst: bpcd version: 07500007

14:07:08.144 [11992.17684] <2> bpcr_get_platform_rqst: Server platform length = 7

14:07:08.363 [11992.17684] <2> bpcr_get_version_rqst: bpcd version: 07500007

14:07:08.628 [11992.17684] <2> bpcr_patch_version_rqst: theRest == > <

14:07:08.628 [11992.17684] <2> bpcr_get_version_rqst: bpcd version: 07500007

14:07:08.877 [11992.17684] <2> bpcr_patch_version_rqst: theRest == > <

14:07:08.877 [11992.17684] <2> bpcr_get_version_rqst: bpcd version: 07500007

PEER_NAME = gxxxxxx2

HOST_NAME = fxxxxxxxx4

CLIENT_NAME = fxxxxxxxx4

VERSION = 0x07500007

PLATFORM = win_x64

PATCH_VERSION = 7.5.0.7

SERVER_PATCH_VERSION = 7.5.0.7

MASTER_SERVER = gxxxxxx2

EMM_SERVER = gxxxxxx2

NB_MACHINE_TYPE = CLIENT

X.X.X.71:4868 <- X.X.X.78:50630

<2>bptestbpcd: EXIT status = 0

14:07:09.205 [11992.17684] <2> bptestbpcd: EXIT status = 0

 C:\Program Files\Veritas\NetBackup\bin>