Netbackup client connectivity issue
bptestbpcd from master server to client is failing with below error, although able to connect all the backup ports and name resolution is fine and also able to issue certificate successfully. Still backups are failing with error 58 can't connect client
vnet_registerPBXServer: ../../libvlibs/vnet_pbx.c.94: pbxRegisterExEx failed with error 110:Connection timed outdaemon_select_and_accept_ex: vnet_registerPBXServer() failed: 47
BPTESTBPCD
/usr/openv/netbackup/bin/admincmd/bptestbpcd -verbose -debug -client XXXXXXXX.XXXX.XXXX
14:32:29.524 [30023] <2> bptestbpcd: VERBOSE = 0
14:32:29.525 [30023] <2> ConnectionCache::connectAndCache: Acquiring new connection for host XXXXXXXX.XX.XXXX, query type 223
14:32:29.528 [30023] <2> logconnections: BPDBM CONNECT FROM XXX.XXX.XXX.XXX TO XXX.XXX.XXX.XXX fd = 3
14:32:29.529 [30023] <2> db_CLIENTsend: reset client protocol version from 0 to 9
14:32:29.533 [30023] <2> db_getCLIENT: db_CLIENTreceive: no entity was found 227
14:32:29.533 [30023] <2> closeConnection: Caching connection for query type 223 for reuse
14:32:29.542 [30023] <2> vnet_pbxConnect_ex: pbxConnectExEx() failed: 104
14:32:29.542 [30023] <2> vnet_pbxConnect_ex: ../../libvlibs/vnet_pbx.c.674: pbxSetAddrEx/pbxConnectExEx return error 104:Connection reset by peer
14:32:29.542 [30023] <8> do_pbx_service: [vnet_connect.c:4012] vnet_pbxConnect() failed, status=18, errno=2, use_vnetd=0, cr->vcr_service=bpcd
14:32:29.542 [30023] <8> async_connect: [vnet_connect.c:3543] do_service failed 18 0x12
14:32:29.544 [30023] <8> async_connect: [vnet_connect.c:3566] getsockopt SO_ERROR returned 111 0x6f
14:32:30.547 [30023] <8> async_connect: [vnet_connect.c:3566] getsockopt SO_ERROR returned 111 0x6f
14:32:32.549 [30023] <8> async_connect: [vnet_connect.c:3566] getsockopt SO_ERROR returned 111 0x6f
14:32:36.551 [30023] <8> async_connect: [vnet_connect.c:3566] getsockopt SO_ERROR returned 111 0x6f
14:32:44.554 [30023] <8> async_connect: [vnet_connect.c:3566] getsockopt SO_ERROR returned 111 0x6f
14:32:44.554 [30023] <16> connect_to_service: connect failed STATUS (18) CONNECT_FAILED status: FAILED, (24) BAD_VERSION; system: (2) No such file or directory; FROM 0.0.0.0 TO XXXXXXXX.XXXX.XXXX XXX.XXX.XXX.XXX bpcd VIA pbx status: FAILED, (42) CONNECT_REFUSED; system: (111) Connection refused; FROM 0.0.0.0 TO XXXXXXXX.XXXX.XXXX XXX.XXX.XXX.XXX bpcd VIA vnetd
14:32:44.554 [30023] <16> connect_to_service: JSON data = {"allow_large_status": {"timestamp": 1672218149, "who": "vnet_tss_init", "line_number": 32, "comment": "allow vnet status > 255", "data": true}, "direct_connect": {"timestamp": 1672218149, "who": "connect_to_service", "line_number": 2199, "comment": "connect parameters", "data": {"who": "vnet_connect_to_bpcd", "host": "XXXXXXXX.XXXX.XXXX", "service": "bpcd", "override_required_interface": null, "extra_tries_on_connect": 0, "getsock_disable_to": 0, "overide_connect_timeout": 0, "connect_options": {"server": null, "callback_kind": {"number": 1, "symbol": "NBCONF_CALLBACK_KIND_VNETD", "description": "Vnetd"}, "daemon_port_type": {"number": 0, "symbol": "NBCONF_DAEMON_PORT_TYPE_AUTOMATIC", "description": "Automatic"}, "reserved_port_kind": {"number": 0, "symbol": "NBCONF_RESERVED_PORT_KIND_LEGACY", "description": "Legacy"}}}}, "status": {"timestamp": 1672218164, "who": "connect_to_service", "line_number": 2465, "comment": "vnet status", "data": 18}, "connect_recs": {"timestamp": 1672218164, "who": "vnet_tss_get", "line_number": 97, "comment": "connect rec status messages", "data": "connect failed STATUS (18) CONNECT_FAILED\n\tstatus: FAILED, (24) BAD_VERSION; system: (2) No such file or directory; FROM 0.0.0.0 TO XXXXXXXX.XXXX.XXXX XXX.XXX.XXX.XXX bpcd VIA pbx\n\tstatus: FAILED, (42) CONNECT_REFUSED; system: (111) Connection refused; FROM 0.0.0.0 TO XXXXXXXX.XXXX.XXXX XXX.XXX.XXX.XXX bpcd VIA vnetd"}}
14:32:44.554 [30023] <8> vnet_connect_to_bpcd: [vnet_connect.c:623] connect_to_service() failed 18 0x12
14:32:44.554 [30023] <16> local_bpcr_connect: vnet_connect_to_bpcd(XXXXXXXX.XXXX.XXXX) failed: 18
14:32:44.554 [30023] <2> local_bpcr_connect: Can't connect to client XXXXXXXX.XXXX.XXXX
14:32:44.554 [30023] <2> ConnectToBPCD: bpcd_connect_and_verify(XXXXXXXX.XXXX.XXXX, XXXXXXXX.XXXX.XXXX) failed: 25
14:32:44.554 [30023] <16> bptestbpcd main: JSON proxy message = {"allow_large_status": {"timestamp": 1672218149, "who": "vnet_tss_init", "line_number": 32, "comment": "allow vnet status > 255", "data": true}, "direct_connect": {"timestamp": 1672218149, "who": "connect_to_service", "line_number": 2199, "comment": "connect parameters", "data": {"who": "vnet_connect_to_bpcd", "host": "XXXXXXXX.XXXX.XXXX", "service": "bpcd", "override_required_interface": null, "extra_tries_on_connect": 0, "getsock_disable_to": 0, "overide_connect_timeout": 0, "connect_options": {"server": null, "callback_kind": {"number": 1, "symbol": "NBCONF_CALLBACK_KIND_VNETD", "description": "Vnetd"}, "daemon_port_type": {"number": 0, "symbol": "NBCONF_DAEMON_PORT_TYPE_AUTOMATIC", "description": "Automatic"}, "reserved_port_kind": {"number": 0, "symbol": "NBCONF_RESERVED_PORT_KIND_LEGACY", "description": "Legacy"}}}}, "status": {"timestamp": 1672218164, "who": "connect_to_service", "line_number": 2465, "comment": "vnet status", "data": 18}, "connect_recs": {"timestamp": 1672218164, "who": "vnet_tss_get", "line_number": 97, "comment": "connect rec status messages", "data": "connect failed STATUS (18) CONNECT_FAILED\n\tstatus: FAILED, (24) BAD_VERSION; system: (2) No such file or directory; FROM 0.0.0.0 TO XXXXXXXX.XXXX.XXXX XXX.XXX.XXX.XXX bpcd VIA pbx\n\tstatus: FAILED, (42) CONNECT_REFUSED; system: (111) Connection refused; FROM 0.0.0.0 TO XXXXXXXX.XXXX.XXXX XXX.XXX.XXX.XXX bpcd VIA vnetd"}}
<16>bptestbpcd main: Function ConnectToBPCD(XXXXXXXX.XXXX.XXXX) failed: 25
14:32:44.554 [30023] <16> bptestbpcd main: Function ConnectToBPCD(XXXXXXXX.XXXX.XXXX) failed: 25
<16>bptestbpcd main: cannot connect on socket
14:32:44.559 [30023] <16> bptestbpcd main: cannot connect on socket
<2>bptestbpcd: cannot connect on socket
14:32:44.559 [30023] <2> bptestbpcd: cannot connect on socket
<2>bptestbpcd: EXIT status = 25
14:32:44.559 [30023] <2> bptestbpcd: EXIT status = 25