08-21-2018 09:13 AM
All the services are running fine, no firewall configuration inbetween, DNS is working fine, frwdlookup and reverse lookup running fine but still connectivity could not be established
I cannot able to terminate the VNET using the command "/usr/openv/netbackup/bin/vnetd -terminate", Its throwing the below error and file also not error also we could able to see in syslog when bptestbpcd is triggered from master server
09:00:32.394 [14389] <2> setup_debug_log: switched debug log file for vnetd
09:00:32.394 [14389] <2> main: VERBOSE = 5
09:00:32.395 [14389] <2> main: [vnetd.c:296] max_time 120 0x78
09:00:32.395 [14389] <2> logparams: /usr/openv/netbackup/bin/vnetd -terminate
09:00:32.395 [14389] <2> vnet_pcache_init_table: [vnet_private.c:235] starting cache size 200 0xc8
09:00:32.395 [14389] <8> file_to_cache_item: [vnet_addrinfo.c:6574] fopen() failed ERRNO=2 FILE=/usr/openv/var/host_cache/1ff/ffffffff+vnetd,1,1,0,1,0+.txt
09:00:32.395 [14389] <2> retry_getaddrinfo: [vnet_addrinfo.c:908] using SVC=vnetd PORT=13724
09:00:32.395 [14389] <2> retry_getaddrinfo: [vnet_addrinfo.c:912] adjusted service name 13724
09:00:32.395 [14389] <2> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1632] found via getaddrinfo NAME=NULL SVC=vnetd
09:00:32.396 [14389] <16> daemon_terminate_local: connect failed: 111
09:00:32.396 [14389] <8> try_terminate: [vnetd.c:2737] daemon_terminate_local() failed 21 0x15
09:00:32.396 [14389] <16> main: EXIT status = 21
Aug 10 06:32:14 xinetd[20028]: execv( /usr/openv/netbackup/localbin/vnetd ) failed: No such file or directory (errno = 2)
Aug 10 06:32:14 xinetd[20028]: execv( /usr/openv/netbackup/localbin/vnetd ) failed: No such file or directory (errno = 2)
08-21-2018 10:31 AM - edited 08-22-2018 04:17 AM
Why are you focussing on vnetd when NBU is firstly using pbx for connectivity?
The path to vnetd looks incorrect. To my knowledge, it is usr/openv/netbackup/bin/vnetd.
Please show us output of :
bptestbpcd -client (name) -verbose -debug
Please ensure that bpcd log folder exists on the client for troubleshooting.
08-22-2018 02:27 AM
was this a new client install or a upgrade install from a older version?
Which version are you running?
08-28-2018 04:29 AM
./bptestbpcd -client QCHATCRMBLD-UBUNTU02
^C
sourshoes{54}> ./bptestbpcd -client QCHATCRMBLD-UBUNTU02 -debug -verbose
04:23:30.926 [5112] <2> bptestbpcd: VERBOSE = 5
04:23:30.929 [5112] <2> ConnectToBPCD: db_getCLIENT(QCHATCRMBLD-UBUNTU02) failed: 227
04:23:30.951 [5112] <8> async_connect: [vnet_connect.c:2081] getsockopt SO_ERROR returned 146 0x92
04:23:30.955 [5112] <2> vnet_version_connect_ex: vnet_vnetd.c.1731: 0: Function failed: 24 0x00000018
04:23:30.955 [5112] <8> do_vnetd_service: [vnet_connect.c:2338] vnet_version_connect failed 24 0x18
04:23:30.955 [5112] <8> async_connect: [vnet_connect.c:2058] do_service failed 24 0x18
04:23:31.962 [5112] <8> async_connect: [vnet_connect.c:2081] getsockopt SO_ERROR returned 146 0x92
04:23:33.971 [5112] <8> async_connect: [vnet_connect.c:2081] getsockopt SO_ERROR returned 146 0x92
04:23:37.972 [5112] <8> async_connect: [vnet_connect.c:2081] getsockopt SO_ERROR returned 146 0x92
04:23:45.980 [5112] <8> async_connect: [vnet_connect.c:2081] getsockopt SO_ERROR returned 146 0x92
04:23:45.981 [5112] <16> connect_to_service: connect failed STATUS (18) CONNECT_FAILED status: FAILED, (42) CONNECT_REFUSED; system: (146) Connection refused; FROM 0.0.0.0 TO QCHATCRMBLD-UBUNTU02 10.52.254.235 bpcd VIA pbx status: FAILED, (24) BAD_VERSION; system: (150) Operation now in progress; FROM 0.0.0.0 TO QCHATCRMBLD-UBUNTU02 10.52.254.235 bpcd VIA vnetd
04:23:45.981 [5112] <16> connect_to_service: JSON data = {"allow_large_status": {"timestamp": 1535455410, "who": "vnet_tss_init", "line_number": 32, "comment": "allow vnet status > 255", "data": true}, "direct_connect": {"timestamp": 1535455410, "who": "connect_to_service", "line_number": 834, "comment": "connect parameters", "data": {"who": "vnet_connect_to_bpcd", "host": "QCHATCRMBLD-UBUNTU02", "service": "bpcd", "override_required_interface": null, "extra_tries_on_connect": 0, "getsock_disable_to": 0, "overide_connect_timeout": 0, "connect_options": {"server": null, "callback_kind": {"number": 1, "symbol": "NBCONF_CALLBACK_KIND_VNETD", "description": "Vnetd"}, "daemon_port_type": {"number": 0, "symbol": "NBCONF_DAEMON_PORT_TYPE_AUTOMATIC", "description": "Automatic"}, "reserved_port_kind": {"number": 0, "symbol": "NBCONF_RESERVED_PORT_KIND_LEGACY", "description": "Legacy"}}}}, "status": {"timestamp": 1535455425, "who": "connect_to_service", "line_number": 981, "comment": "vnet status", "data": 18}, "connect_recs": {"timestamp": 1535455425, "who": "vnet_tss_get", "line_number": 97, "comment": "connect rec status messages", "data": "connect failed STATUS (18) CONNECT_FAILED\n\tstatus: FAILED, (42) CONNECT_REFUSED; system: (146) Connection refused; FROM 0.0.0.0 TO QCHATCRMBLD-UBUNTU02 10.52.254.235 bpcd VIA pbx\n\tstatus: FAILED, (24) BAD_VERSION; system: (150) Operation now in progress; FROM 0.0.0.0 TO QCHATCRMBLD-UBUNTU02 10.52.254.235 bpcd VIA vnetd"}}
04:23:45.981 [5112] <8> vnet_connect_to_bpcd: [vnet_connect.c:565] connect_to_service() failed 18 0x12
04:23:45.981 [5112] <16> local_bpcr_connect: vnet_connect_to_bpcd(QCHATCRMBLD-UBUNTU02) failed: 18
04:23:45.982 [5112] <2> local_bpcr_connect: Can't connect to client QCHATCRMBLD-UBUNTU02
04:23:45.982 [5112] <2> ConnectToBPCD: bpcd_connect_and_verify(QCHATCRMBLD-UBUNTU02, QCHATCRMBLD-UBUNTU02) failed: 25
04:23:45.982 [5112] <16> bptestbpcd main: JSON proxy message = {"allow_large_status": {"timestamp": 1535455410, "who": "vnet_tss_init", "line_number": 32, "comment": "allow vnet status > 255", "data": true}, "direct_connect": {"timestamp": 1535455410, "who": "connect_to_service", "line_number": 834, "comment": "connect parameters", "data": {"who": "vnet_connect_to_bpcd", "host": "QCHATCRMBLD-UBUNTU02", "service": "bpcd", "override_required_interface": null, "extra_tries_on_connect": 0, "getsock_disable_to": 0, "overide_connect_timeout": 0, "connect_options": {"server": null, "callback_kind": {"number": 1, "symbol": "NBCONF_CALLBACK_KIND_VNETD", "description": "Vnetd"}, "daemon_port_type": {"number": 0, "symbol": "NBCONF_DAEMON_PORT_TYPE_AUTOMATIC", "description": "Automatic"}, "reserved_port_kind": {"number": 0, "symbol": "NBCONF_RESERVED_PORT_KIND_LEGACY", "description": "Legacy"}}}}, "status": {"timestamp": 1535455425, "who": "connect_to_service", "line_number": 981, "comment": "vnet status", "data": 18}, "connect_recs": {"timestamp": 1535455425, "who": "vnet_tss_get", "line_number": 97, "comment": "connect rec status messages", "data": "connect failed STATUS (18) CONNECT_FAILED\n\tstatus: FAILED, (42) CONNECT_REFUSED; system: (146) Connection refused; FROM 0.0.0.0 TO QCHATCRMBLD-UBUNTU02 10.52.254.235 bpcd VIA pbx\n\tstatus: FAILED, (24) BAD_VERSION; system: (150) Operation now in progress; FROM 0.0.0.0 TO QCHATCRMBLD-UBUNTU02 10.52.254.235 bpcd VIA vnetd"}}
<16>bptestbpcd main: Function ConnectToBPCD(QCHATCRMBLD-UBUNTU02) failed: 25
04:23:45.983 [5112] <16> bptestbpcd main: Function ConnectToBPCD(QCHATCRMBLD-UBUNTU02) failed: 25
<16>bptestbpcd main: cannot connect on socket
04:23:46.001 [5112] <16> bptestbpcd main: cannot connect on socket
<2>bptestbpcd: cannot connect on socket
04:23:46.002 [5112] <2> bptestbpcd: cannot connect on socket
<2>bptestbpcd: EXIT status = 25
04:23:46.002 [5112] <2> bptestbpcd: EXIT status = 25
cannot connect on socket
08-28-2018 04:31 AM
Client details
NBU version 7.5.0.5
OS Ubunto 12.04
08-28-2018 04:46 AM
NBU 8.0 is supported on Ubuntu 12.04.
Can you try to upgrade the client to same NBU version as master/media server(s)?
Please also check that all of these processes are running on the client:
/usr/openv/netbackup/bin/vnetd -standalone
/usr/openv/netbackup/bin/bpcd -standalone
/opt/VRTSpbx/bin/pbx_exchange
Remember to create bpcd log folder on the client before next connection attempt.
If NBU is running on the client and there is no firewall blocking connections, all should be good and connection attempt will be logged in client's bpcd log.
08-28-2018 11:39 PM
Have you tried to telnet to the client on 1556 & 13724 from the master ? It of course requires the netbackup processes are running on the client.
I my experience most Linux distrobutions has a builtin firewall, get the lInux admin to verify that it is disabled.
09-05-2018 12:27 PM
Please find the below vnetd logs
vnetd logging
Server name{117}> cat root.090518_00001.log
13:20:08.458 [13523.13523] <2> setup_debug_log: switched debug log file for vnetd
13:20:08.460 [13523.13523] <4> main: VERBOSE=0, VNETD VERBOSE=0 Timestamp=1536174098
13:20:08.461 [13523.13523] <2> logparams: /usr/openv/netbackup/bin/vnetd -standalone
13:20:08.461 [13525.13525] <2> setup_debug_log: switching debug log file for vnetd
13:20:08.462 [13525.13525] <2> setup_debug_log: switched debug log file for vnetd
13:20:08.462 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:09.462 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:10.463 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:11.463 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:12.463 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:13.464 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:14.464 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:15.464 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:16.464 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:17.464 [13525.13525] <2> legacy_listen: bind(13724) failed: 98
13:20:17.464 [13525.13525] <2> daemon_startup_listeners: 4 legacy_listen(vnetd) failed: 21
13:20:17.504 [13525.13525] <2> vnet_registerPBXServer: ../../libvlibs/vnet_pbx.c.110: pbxRegisterEx failed with error 101:Network is unreachable
13:20:17.504 [13525.13525] <16> daemon_startup_listeners: registering service vnetd with pbx failed: 47
13:20:17.504 [13525.13525] <2> daemon_startup_listeners: Continuing despite PBX registration error: 21
13:20:17.504 [13525.13525] <2> daemon_startup_listeners: no listening sockets open vnetd NULL vnetd
13:20:17.504 [13525.13525] <8> main: [vnetd.c:489] daemon_startup_listeners failed 21 0x15
13:20:17.505 [13525.13525] <16> main: EXIT status = 21
serevr naem{118}> sudo /usr/openv/netbackup/bin/vnetd -terminate -debug
13:21:42.833 [16250.16250] <16> daemon_terminate_local: connect failed: 111
13:21:42.833 [16250.16250] <8> try_terminate: [vnetd.c:2868] daemon_terminate_local() failed 21 0x15
EXIT status = 21
13:21:42.833 [16250.16250] <16> main: EXIT status = 21
Even the pbx registration is getting failed it seems
09-05-2018 02:07 PM
09-05-2018 02:24 PM
I have alreafdy tried upgrading machine to 7.7.3 but still no luck
All the ports are communicating and no firewall inbetween, Still this issue occurs
machine is looking for vnet in /usr/openv/netbackup/localbin and vnetd in the /usr/openv/netbackup/bin is not getting started and not showing up in bpps also