Backup failing with error code 59

[Varma_Chiluvuri]

The backup was failing with error code 59

05/13/2014 10:29:16 - granted resource  I00523

05/13/2014 10:29:16 - granted resource  STK.T10000A.006
05/13/2014 10:29:16 - granted resource  grvs01l2-nb-hcart3-robot-tld-3
05/13/2014 10:29:16 - estimated 9655261 kbytes needed
05/13/2014 10:29:16 - Info nbjm (pid=8323110) started backup (backupid=sapi6pdrv-nb_1399991356) job for client sapi6pdrv-nb, policy Granville_Zones_1, schedule Daily on storage unit grvs01l2-nb-hcart3-robot-tld-3
05/13/2014 10:29:17 - started process bpbrm (pid=13825)
05/13/2014 10:29:18 - Info bpbrm (pid=13825) starting bptm
05/13/2014 10:29:18 - Info bpbrm (pid=13825) Started media manager using bpcd successfully
05/13/2014 10:29:19 - Error bpbrm (pid=13825) bpcd on sapi6pdrv-nb exited with status 59: access to the client was not allowed
05/13/2014 10:29:19 - end writing
access to the client was not allowed  (59)

From Master server I have checked communication and it is working fine.

# bpclntcmd -hn sapi6pdrv-nb

host sapi6pdrv-nb: sapi6pdrv at 100.7.0.228
aliases:     sapi6pdrv-nb     sapi6pdrv     100.7.0.228
[root@sun107] /root
#
[root@sun107] /root
# bpclntcmd -ip 100.7.0.228
host 100.7.0.228: sapi6pdrv at 100.7.0.228
aliases:     sapi6pdrv     100.7.0.228

From Client also communication looks good

sapi6pv:root> ./bpclntcmd -pn

expecting response from server sun107-nb
sapi6pdrv sapi6pdrv-nb 100.7.0.228 58764
sapi6pv:root>
sapi6pv:root> ./bpclntcmd -hn sun107-nb
host sun107-nb: sun107 at 100.6.1.107
aliases:     sun107-nb     sun107     100.6.1.107
sapi6pv:root>

[Marianne]

Client is missing a SERVER entry for the media server grvs01l2-nb.

Please create bpcd log folder on the client under ...netbackup/logs.

Retry the backup (or use bptestbpcd on media server grvs01l2-nb).

Check bpcd log on the client to see incoming request from media server IP address.
Next you will see attempt to resolve IP to a hostname.

After this, the client will compare resolved name with known SERVERS.

Add the resolved name of media server to Client's Server list in bp.conf. 
Media server name must be below Master server's entry.
This name must match the hostname seen in bpcd log EXACTLY.

[Varma_Chiluvuri]

Hi Marianne

Please find the log details below

BPTESTBPCD

11:05:05.200 [13691] <2> vnet_pbxAcceptSocket: Accepted sock[11] from 10.48.184.190:55380
11:05:05.201 [13691] <2> bpcd main: accept sock = 11
11:05:05.209 [20348] <2> process_requests: offset to GMT 18000
11:05:05.210 [20348] <2> logconnections: BPCD ACCEPT FROM 10.48.184.190.55380 TO 100.7.0.228.1556 fd = 0
11:05:05.212 [20348] <2> process_requests: setup_sockopts complete
11:05:05.217 [20348] <2> bpcd peer_hostname: Connection from host nbu-media-grv (10.48.184.190) port 55380
11:05:05.218 [20348] <2> bpcd valid_server: comparing sun107-nb and nbu-media-grv
11:05:05.223 [20348] <2> bpcd valid_server: comparing nbu-media-grv-nb and nbu-media-grv
11:05:05.224 [20348] <4> bpcd valid_server: hostname comparison succeeded
11:05:05.227 [20348] <2> process_requests: output socket port number = 1
11:05:05.531 [20348] <8> verify_hashes: [vnet_vnetd.c:1641] hash_str dcba10a5549025bb4a837cfd3c4c9468
11:05:05.531 [20348] <2> process_requests: Duplicated vnetd socket on stderr
11:05:05.531 [20348] <2> process_requests: <---- NetBackup 7.5 0 ------------initiated
11:05:05.531 [20348] <2> process_requests: VERBOSE = 0
11:05:05.531 [20348] <2> process_requests: Not using VxSS authentication with nbu-media-grv
11:05:05.738 [20348] <2> process_requests: BPCD_GET_STDOUT_SOCKET_RQST
11:05:05.738 [20348] <2> process_requests: socket port number = 1
11:05:05.933 [20348] <8> verify_hashes: [vnet_vnetd.c:1641] hash_str b939a8ce936191d63ffd4d48afa51ef4
11:05:05.933 [20348] <2> process_requests: Connected on output socket
11:05:05.933 [20348] <2> process_requests: Skipping shutdown of send side of stdout.
11:05:05.933 [20348] <2> process_requests: Duplicated socket on stdout
11:05:05.934 [20348] <2> process_requests: BPCD_DISCONNECT_RQST
11:05:05.934 [20348] <2> bpcd exit_bpcd: exit status 0  ----------->exiting

TRIGGERED BACKUP

11:07:31.117 [21180] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1273] vnet_cached_getaddrinfo_and_update() failed 6 0x6
11:07:31.117 [21180] <8> vnet_same_host_and_update: [vnet_addrinfo.c:2832] vnet_cached_getaddrinfo() failed STAT=6 RV=8 NAME1=sape6pdrv-nb
11:07:31.117 [21180] <4> bpcd valid_server: 14.4.2.190 is not a master server
11:07:31.117 [21180] <16> bpcd valid_server: 14.4.2.190 is not a media server either
11:07:31.118 [21180] <2> process_requests: output socket port number = 1
11:07:31.250 [21180] <8> verify_hashes: [vnet_vnetd.c:1641] hash_str 09569844175378c24b5e422269156f59
11:07:31.250 [21180] <2> process_requests: Duplicated vnetd socket on stderr
11:07:31.250 [21180] <2> process_requests: <---- NetBackup 7.5 0 ------------initiated
11:07:31.250 [21180] <2> process_requests: VERBOSE = 0
11:07:31.251 [21180] <2> bpcd exit_bpcd: exit status 46  ----------->exiting
11:07:31.251 [21180] <4> bpcd exit_bpcd: FTL - BPCD EXIT STATUS 46

11:07:31.251 [21180] <16> process_requests: Server access denied

[RamNagalla]

did you run the bptestbpcd command from the same media server that is being used for backups?

please post the bpcd log from the client 

[rk1074]

please provide the output of bp.conf file from the client

/usr/openv/netbackup/bp.conf

[Marianne · Accepted solution]

If we look at Storage Unit name in failed job, the media server name seems to be grvs01l2-nb. You need to do bptestbpcd from this media server. One more thing: bpcd log shows incoming IP address 14.4.2.190. client can clearly not resolve this IP to a hostname. Add this IP and media server hostname to client's /etc/hosts file as well as SERVER entry in client's bp.conf.

[Varma_Chiluvuri]

@Marianne 

I have tried changing the media server, please find the log for nbu-media-grv media server

05/13/2014 11:07:29 - granted resource  nbu-media-grv-nb-hcart3-robot-tld-3
05/13/2014 11:07:29 - estimated 120341179 kbytes needed
05/13/2014 11:07:29 - Info nbjm (pid=8323110) started backup (backupid=sapi6pdrv-nb_1399993649) job for client sapi6pdrv-nb, policy Granville_Zones_1, schedule Weekly on storage unit nbu-media-grv-nb-hcart3-robot-tld-3
05/13/2014 11:07:29 - started process bpbrm (pid=16908370)
05/13/2014 11:07:30 - Info bpbrm (pid=16908370) starting bptm
05/13/2014 11:07:30 - Info bpbrm (pid=16908370) Started media manager using bpcd successfully
05/13/2014 11:07:31 - Error bpbrm (pid=16908370) bpcd on sapi6pdrv-nb exited with status 59: access to the client was not allowed
05/13/2014 11:07:31 - end writing
access to the client was not allowed  (59)

@rk1074 

SERVER = sun107-nb
SERVER = nbu-media-grv-nb

CLIENT_NAME = sapi6pdrv-nb

[Varma_Chiluvuri]

Thanks Marianne,

I have added 14.2.2.190 IP (this is backup IP of media server not the public IP) in client /etc/hosts file.

This fixed the issue.

[Varma_Chiluvuri]

Marianne,

I have one question, for many other servers we don't have media server backup IP entry in /etc/hosts file. But only for this server 14.*.*.* IP is trying to access.

Can you clear me on this ?

[Marianne]

Check if client is on DNS.

Do
nslookup <Media-server-name>
and
nslookup <Media-server-IP-address>

If this fails, it means client is either not on DNS or else no DNS entries exist for media server's backup IP and hostname.