Can't connect to UNIX server

[smwoodcrafts]

I am having a problem with a UNIX backup. It is though a firewall, but I don't think that is the issue, I checked. I can't connect through host properties and the backup fails with a 58 error. I had the SA create a bpcd log directory and started a backup job. This is the log file after 2 attempts and a bptestbpcd command.


This is Client side:

11:24:06.301 [19310] <2> bpcd main: offset to GMT 18000
11:24:06.302 [19310] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.903 TO 10.1.59.44.13782
11:24:06.302 [19310] <2> bpcd main: setup_sockopts complete
11:24:06.306 [19310] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 903
11:24:06.306 [19310] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:24:06.307 [19310] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:24:06.307 [19310] <4> bpcd valid_server: hostname comparison succeeded
11:24:06.308 [19310] <2> bpcd main: output socket port number = 13782
11:25:19.273 [19350] <2> bpcd main: offset to GMT 18000
11:25:19.273 [19350] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.812 TO 10.1.59.44.13782
11:25:19.274 [19350] <2> bpcd main: setup_sockopts complete
11:25:19.278 [19350] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 812
11:25:19.278 [19350] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:25:19.287 [19350] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:25:19.287 [19350] <4> bpcd valid_server: hostname comparison succeeded
11:25:19.288 [19350] <2> bpcd main: output socket port number = 13782
11:27:21.131 [19660] <2> bpcd main: offset to GMT 18000
11:27:21.132 [19660] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.722 TO 10.1.59.44.13782
11:27:21.132 [19660] <2> bpcd main: setup_sockopts complete
11:27:21.137 [19660] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 722
11:27:21.137 [19660] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:27:21.137 [19660] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:27:21.138 [19660] <4> bpcd valid_server: hostname comparison succeeded
11:27:21.138 [19660] <2> bpcd main: output socket port number = 13782
11:39:24.313 [19310] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
11:39:24.314 [19310] <16> bpcd main: get_vnetd_socket failed: 25
11:40:37.294 [19350] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
11:40:37.295 [19350] <16> bpcd main: get_vnetd_socket failed: 25
11:42:39.156 [19660] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
11:42:39.156 [19660] <16> bpcd main: get_vnetd_socket failed: 25
11:51:02.485 [21486] <2> bpcd main: offset to GMT 18000
11:51:02.486 [21486] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.1020 TO 10.1.59.44.13782
11:51:02.486 [21486] <2> bpcd main: setup_sockopts complete
11:51:02.490 [21486] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 1020
11:51:02.490 [21486] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:51:02.491 [21486] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:51:02.491 [21486] <4> bpcd valid_server: hostname comparison succeeded
11:51:02.492 [21486] <2> bpcd main: output socket port number = 13782
12:06:20.555 [21486] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
12:06:20.555 [21486] <16> bpcd main: get_vnetd_socket failed: 25
12:16:50.276 [23821] <2> bpcd main: offset to GMT 18000
12:16:50.276 [23821] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.665 TO 10.1.59.44.13782
12:16:50.277 [23821] <2> bpcd main: setup_sockopts complete
12:16:50.281 [23821] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 665
12:16:50.281 [23821] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
12:16:50.282 [23821] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
12:16:50.282 [23821] <4> bpcd valid_server: hostname comparison succeeded
12:16:50.282 [23821] <2> bpcd main: output socket port number = 13782
12:32:08.296 [23821] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
12:32:08.296 [23821] <16> bpcd main: get_vnetd_socket failed: 25
12:42:17.521 [26001] <2> bpcd main: offset to GMT 18000
12:42:17.522 [26001] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.518 TO 10.1.59.44.13782
12:42:17.522 [26001] <2> bpcd main: setup_sockopts complete
12:42:17.526 [26001] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 518
12:42:17.538 [26001] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
12:42:17.539 [26001] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
12:42:17.539 [26001] <4> bpcd valid_server: hostname comparison succeeded
12:42:17.539 [26001] <2> bpcd main: output socket port number = 13782
12:57:35.577 [26001] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
12:57:35.577 [26001] <16> bpcd main: get_vnetd_socket failed: 25
13:08:00.730 [28110] <2> bpcd main: offset to GMT 18000
13:08:00.731 [28110] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.889 TO 10.1.59.44.13782
13:08:00.731 [28110] <2> bpcd main: setup_sockopts complete
13:08:00.735 [28110] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 889
13:08:00.735 [28110] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:08:00.736 [28110] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:08:00.736 [28110] <4> bpcd valid_server: hostname comparison succeeded
13:08:00.762 [28110] <2> bpcd main: output socket port number = 13782
13:22:16.498 [29350] <2> bpcd main: offset to GMT 18000
13:22:16.499 [29350] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.600 TO 10.1.59.44.13782
13:22:16.499 [29350] <2> bpcd main: setup_sockopts complete
13:22:16.503 [29350] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 600
13:22:16.503 [29350] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:22:16.504 [29350] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:22:16.504 [29350] <4> bpcd valid_server: hostname comparison succeeded
13:22:16.505 [29350] <2> bpcd main: output socket port number = 13782
13:23:18.768 [28110] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
13:23:18.768 [28110] <16> bpcd main: get_vnetd_socket failed: 25
13:30:31.304 [29988] <2> bpcd main: offset to GMT 18000
13:30:31.305 [29988] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.778 TO 10.1.59.44.13782
13:30:31.305 [29988] <2> bpcd main: setup_sockopts complete
13:30:31.309 [29988] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 778
13:30:31.309 [29988] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:30:31.310 [29988] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:30:31.310 [29988] <4> bpcd valid_server: hostname comparison succeeded
13:30:31.311 [29988] <2> bpcd main: output socket port number = 13782
13:32:57.348 [318] <2> bpcd main: offset to GMT 18000
13:32:57.349 [318] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.545 TO 10.1.59.44.13782
13:32:57.349 [318] <2> bpcd main: setup_sockopts complete
13:32:57.353 [318] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 545
13:32:57.353 [318] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:32:57.354 [318] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:32:57.354 [318] <4> bpcd valid_server: hostname comparison succeeded
13:32:57.355 [318] <2> bpcd main: output socket port number = 13782
13:37:34.570 [29350] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
13:37:34.570 [29350] <16> bpcd main: get_vnetd_socket failed: 25

This is the output of the bptestbpcd command on the master with -verbose and -debug added.

11:24:06.301 [19310] <2> bpcd main: offset to GMT 18000
11:24:06.302 [19310] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.903 TO 10.1.59.44.13782
11:24:06.302 [19310] <2> bpcd main: setup_sockopts complete
11:24:06.306 [19310] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 903
11:24:06.306 [19310] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:24:06.307 [19310] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:24:06.307 [19310] <4> bpcd valid_server: hostname comparison succeeded
11:24:06.308 [19310] <2> bpcd main: output socket port number = 13782
11:25:19.273 [19350] <2> bpcd main: offset to GMT 18000
11:25:19.273 [19350] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.812 TO 10.1.59.44.13782
11:25:19.274 [19350] <2> bpcd main: setup_sockopts complete
11:25:19.278 [19350] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 812
11:25:19.278 [19350] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:25:19.287 [19350] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:25:19.287 [19350] <4> bpcd valid_server: hostname comparison succeeded
11:25:19.288 [19350] <2> bpcd main: output socket port number = 13782
11:27:21.131 [19660] <2> bpcd main: offset to GMT 18000
11:27:21.132 [19660] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.722 TO 10.1.59.44.13782
11:27:21.132 [19660] <2> bpcd main: setup_sockopts complete
11:27:21.137 [19660] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 722
11:27:21.137 [19660] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:27:21.137 [19660] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:27:21.138 [19660] <4> bpcd valid_server: hostname comparison succeeded
11:27:21.138 [19660] <2> bpcd main: output socket port number = 13782
11:39:24.313 [19310] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
11:39:24.314 [19310] <16> bpcd main: get_vnetd_socket failed: 25
11:40:37.294 [19350] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
11:40:37.295 [19350] <16> bpcd main: get_vnetd_socket failed: 25
11:42:39.156 [19660] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
11:42:39.156 [19660] <16> bpcd main: get_vnetd_socket failed: 25
11:51:02.485 [21486] <2> bpcd main: offset to GMT 18000
11:51:02.486 [21486] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.1020 TO 10.1.59.44.13782
11:51:02.486 [21486] <2> bpcd main: setup_sockopts complete
11:51:02.490 [21486] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 1020
11:51:02.490 [21486] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
11:51:02.491 [21486] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
11:51:02.491 [21486] <4> bpcd valid_server: hostname comparison succeeded
11:51:02.492 [21486] <2> bpcd main: output socket port number = 13782
12:06:20.555 [21486] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
12:06:20.555 [21486] <16> bpcd main: get_vnetd_socket failed: 25
12:16:50.276 [23821] <2> bpcd main: offset to GMT 18000
12:16:50.276 [23821] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.665 TO 10.1.59.44.13782
12:16:50.277 [23821] <2> bpcd main: setup_sockopts complete
12:16:50.281 [23821] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 665
12:16:50.281 [23821] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
12:16:50.282 [23821] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
12:16:50.282 [23821] <4> bpcd valid_server: hostname comparison succeeded
12:16:50.282 [23821] <2> bpcd main: output socket port number = 13782
12:32:08.296 [23821] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
12:32:08.296 [23821] <16> bpcd main: get_vnetd_socket failed: 25
12:42:17.521 [26001] <2> bpcd main: offset to GMT 18000
12:42:17.522 [26001] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.518 TO 10.1.59.44.13782
12:42:17.522 [26001] <2> bpcd main: setup_sockopts complete
12:42:17.526 [26001] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 518
12:42:17.538 [26001] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
12:42:17.539 [26001] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
12:42:17.539 [26001] <4> bpcd valid_server: hostname comparison succeeded
12:42:17.539 [26001] <2> bpcd main: output socket port number = 13782
12:57:35.577 [26001] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
12:57:35.577 [26001] <16> bpcd main: get_vnetd_socket failed: 25
13:08:00.730 [28110] <2> bpcd main: offset to GMT 18000
13:08:00.731 [28110] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.889 TO 10.1.59.44.13782
13:08:00.731 [28110] <2> bpcd main: setup_sockopts complete
13:08:00.735 [28110] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 889
13:08:00.735 [28110] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:08:00.736 [28110] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:08:00.736 [28110] <4> bpcd valid_server: hostname comparison succeeded
13:08:00.762 [28110] <2> bpcd main: output socket port number = 13782
13:22:16.498 [29350] <2> bpcd main: offset to GMT 18000
13:22:16.499 [29350] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.600 TO 10.1.59.44.13782
13:22:16.499 [29350] <2> bpcd main: setup_sockopts complete
13:22:16.503 [29350] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 600
13:22:16.503 [29350] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:22:16.504 [29350] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:22:16.504 [29350] <4> bpcd valid_server: hostname comparison succeeded
13:22:16.505 [29350] <2> bpcd main: output socket port number = 13782
13:23:18.768 [28110] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
13:23:18.768 [28110] <16> bpcd main: get_vnetd_socket failed: 25
13:30:31.304 [29988] <2> bpcd main: offset to GMT 18000
13:30:31.305 [29988] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.778 TO 10.1.59.44.13782
13:30:31.305 [29988] <2> bpcd main: setup_sockopts complete
13:30:31.309 [29988] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 778
13:30:31.309 [29988] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:30:31.310 [29988] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:30:31.310 [29988] <4> bpcd valid_server: hostname comparison succeeded
13:30:31.311 [29988] <2> bpcd main: output socket port number = 13782
13:32:57.348 [318] <2> bpcd main: offset to GMT 18000
13:32:57.349 [318] <2> logconnections: BPCD ACCEPT FROM 10.1.133.2.545 TO 10.1.59.44.13782
13:32:57.349 [318] <2> bpcd main: setup_sockopts complete
13:32:57.353 [318] <2> bpcd peer_hostname: Connection from host ctcbackup00_nb (10.1.133.2) port 545
13:32:57.353 [318] <2> bpcd valid_server: comparing ctcbackup00 and ctcbackup00_nb
13:32:57.354 [318] <2> bpcd valid_server: comparing ctcbackup00_nb and ctcbackup00_nb
13:32:57.354 [318] <4> bpcd valid_server: hostname comparison succeeded
13:32:57.355 [318] <2> bpcd main: output socket port number = 13782
13:37:34.570 [29350] <16> get_vnetd_socket: vnet_end_connect_back failed: 10
13:37:34.570 [29350] <16> bpcd main: get_vnetd_socket failed: 25

Backup server are running Windows 2003 standard and running Netbackup 6.5.2. I do know that, on the UNIX client, they are using inetd.

Can you help please.

smwoodcrafts

[rjrumfelt]

you've checked and inetd is in fact running on the client.

It looks like the client is having trouble getting back to the master server.  Can you telnet to the master server on port 13724 from the client?

[Marianne]

I agree with Jason.
Have you confirmed that vnetd port (13724) is open in BOTH directions?
Client seems to be unable to connect back: get_vnetd_socket: vnet_end_connect_back failed: 10

[Andy_Welburn]

e.g. for our Solaris 10 clients

# netstat -a | egrep "bpcd|vnetd"
      *.bpcd               *.*                0      0 49152      0 LISTEN
      *.vnetd              *.*                0      0 49152      0 LISTEN

# netstat -na | grep 13724
      *.13724              *.*                0      0 49152      0 LISTEN

# svcs | egrep "bpcd|vnetd|inetd"
online         Feb_13   svc:/network/inetd:default
online         Feb_13   svc:/network/bpcd/tcp:default
online         Feb_13   svc:/network/vnetd/tcp:default

[smwoodcrafts · Accepted solution]

It actually was a firewall issue. The SA said he could connect via 13724, now he can't. I can now connect but am getting a 41 error. I had them create log directory for bpbkar and this is the output for that.

13:53:15.574 [3928] **WARNING** Locale definition file </usr/openv/msg/.conf> no
t found.  Using defaults
13:53:15.576 [3928] <8> bpbkar: WRN - Unsupported parameter: -blks_per_buffer 12
7
13:53:15.579 [3928] <2> logparams: bpbkar32 -r 259200 -ru root -dt 0 -to 0 -clnt
cortes -class TEST_STANDARD -sched Daily -st INCR -bpstart_to 300 -bpend_to 300
-read_to 1200 -blks_per_buffer 127 -use_otm -b cortes_1274291588 -kl 28
13:53:15.593 [3928] <4> bpbkar: INF - Setting network send buffer size to 32032
bytes
13:53:15.593 [3928] <4> bpbkar: INF - setenv KEYWORD=NONE
13:53:15.593 [3928] <4> bpbkar: INF - setenv STREAM_PID=3928
13:53:15.593 [3928] <4> bpbkar: INF - setenv STREAM_NUMBER=0
13:53:15.594 [3928] <4> bpbkar: INF - setenv STREAM_COUNT=0
13:53:15.594 [3928] <4> bpbkar: INF - setenv STREAMS=0
13:53:15.594 [3928] <4> bpbkar: INF - setenv BPSTART_TIMEOUT=300
13:53:15.594 [3928] <4> bpbkar: INF - setenv BPEND_TIMEOUT=300
13:53:15.594 [3928] <4> bpbkar: INF - setenv BACKUPID=cortes_1274291588
13:53:15.594 [3928] <4> bpbkar: INF - setenv UNIXBACKUPTIME=1274291588
13:53:15.595 [3928] <4> bpbkar: INF - setenv BACKUPTIME=Wed May 19 13:53:08 2010
 
13:53:15.596 [3928] <4> bpbkar: INF - BACKUP START
13:53:15.596 [3928] <4> bpbkar: INF - Estimate:-1 -1
13:53:15.626 [3928] <4> bpbkar: INF - Processing /


This is a Solaris 2.6 box

[Marianne]

You mentioned that the NBU version on the master is 6.5.2, but you did not mention the NBU version on the client? I have just checked compatibilty guides - the last NBU version to support Solaris 2.6 was 4.5 (with '98' in Notes column: 98. NetBackup 5.0 and later will not support this OS Version.)

Netbackup only supports one version back - 6.5.x servers will work with 5.x clients.

If I'm not mistaken, vnetd was introduced with 4.5 FP3. You can see in the bpcd log that the connection is made directly to bpcd (13782). If it's pre-FP3 there is probably no vnetd. The default instruction from the server is to connect back on vnetd. (Until 5.x. the default connect-back was on the reserved range of 512 - 1023.)
The vnetd error in bpcd log then actually makes sense: get_vnetd_socket failed
Please check the NBU version:
cat /usr/openv/netbackup/bin/version.
Also use the netstat command that Andy posted to check if vnetd is listening. Check /etc/inetd.conf and /etc/services for existense of vnetd.

Your chances to get this to work without vnetd and version 4.5 are VERY slim...

[rjrumfelt]

I'm assuming you got with the network admin and worked it out?