Solved: Master server reject connections-Adding trusted se...

Rditi · ‎04-06-2015

Dear all,

I'm trying to add a trusted master server but the following warning appears:

Although the master server specified is now identified as a trusted master server , a secure exchange was not made between these two hosts. This is because one of the master servers is rejecting connections. Please refer to the logs for further troubleshooting. After resolving the issue you need to reconfigure the trusted server. (5680)

I cann add a replication target server but doesn't allow to create SLP replication (access is denied 117).

below is the bptestbpcd log

[root@server1 bpcd]# bptestbpcd -verbose -M server2 -debug
23:14:16.618 [5628] <2> bptestbpcd: VERBOSE = 5
23:14:16.622 [5628] <2> vnet_pbxConnect: pbxConnectEx Succeeded
23:14:16.622 [5628] <2> logconnections: BPCD CONNECT FROM 11.5.14.2.37857 TO 11.5.11.4.1556 fd = 3
23:14:16.623 [5628] <2> vnet_pbxConnect: pbxConnectEx Succeeded
23:14:16.626 [5628] <8> do_pbx_service: [vnet_connect.c:2152] via PBX VNETD CONNECT FROM 11.5.14.2.47416 TO 11.5.11.4.1556 fd = 4
23:14:16.626 [5628] <8> vnet_vnetd_connect_forward_socket_begin: [vnet_vnetd.c:443] VN_REQUEST_CONNECT_FORWARD_SOCKET 10 0xa
23:14:16.667 [5628] <8> vnet_vnetd_connect_forward_socket_begin: [vnet_vnetd.c:460] ipc_string /usr/openv/var/tmp/vnet-42257428354848752212000000000-hWrISe
23:14:16.713 [5628] <2> ConnectToBPCD: bpcd_connect_and_verify(server2, server2) failed: 46
23:14:16.714 [5628] <2> vnet_pbxConnect: pbxConnectEx Succeeded
23:14:16.714 [5628] <2> logconnections: BPCD CONNECT FROM 11.5.14.2.54410 TO 11.5.11.4.1556 fd = 3
23:14:16.715 [5628] <2> vnet_pbxConnect: pbxConnectEx Succeeded
23:14:16.716 [5628] <8> do_pbx_service: [vnet_connect.c:2152] via PBX VNETD CONNECT FROM 11.5.14.2.36672 TO 11.5.11.4.1556 fd = 4
23:14:16.716 [5628] <8> vnet_vnetd_connect_forward_socket_begin: [vnet_vnetd.c:443] VN_REQUEST_CONNECT_FORWARD_SOCKET 10 0xa
23:14:16.756 [5628] <8> vnet_vnetd_connect_forward_socket_begin: [vnet_vnetd.c:460] ipc_string /usr/openv/var/tmp/vnet-42262428354848841259000000000-e3PFhl
23:14:16.802 [5628] <2> local_bpcr_connect: got an exit status error message: <?xml version="1.0" encoding="utf-8"?><StatusMsgDoc><StatusMsg NBUStatus="46" Severity="2" VXULoid="137" VXULtid="114">bck1oam11.5.14.2server211.5.11.4<MsgString>Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.</MsgString><StatusMsg NBUStatus="46" Severity="0" VXULoid="137" VXULtid="115">bck1oam11.5.14.2server2<MsgString>Host bck1oam (11.5.14.2) is not an authorized server for host server2.</MsgString></StatusMsg></StatusMsg></StatusMsgDoc>
23:14:16.802 [5628] <2> ConnectToBPCD: bpcd_connect_and_verify(server2, server2) failed: 46
23:14:16.808 [5628] <2> InternalBuffer: <?xml version="1.0" encoding="utf-8"?><StatusMsgDoc><StatusMsg NBUStatus="46" Se
23:14:16.808 [5628] <2> InternalBuffer: verity="2" VXULoid="137" VXULtid="114">bck1oam10.5.149
23:14:16.808 [5628] <2> InternalBuffer: .2server211.5.11.4<MsgString>Request f
23:14:16.809 [5628] <2> InternalBuffer: rom host bck1oam (11.5.14.2) to host server2 (10
23:14:16.809 [5628] <2> InternalBuffer: .5.118.4) is not allowed access.</MsgString><StatusMsg NBUStatus
23:14:16.809 [5628] <2> InternalBuffer: ="46" Severity="0" VXULoid="137" VXULtid="115">bck1oam
23:14:16.809 [5628] <2> InternalBuffer: 11.5.14.2server2<MsgString>Host bck1oam (10.5.149
23:14:16.809 [5628] <2> InternalBuffer: .2) is not an authorized server for host bckserv
23:14:16.809 [5628] <2> InternalBuffer: er2.</MsgString></StatusMsg></StatusMsg></StatusMsgDoc>
<16>bptestbpcd main: Function ConnectToBPCD(server1) failed: 46
23:14:16.809 [5628] <16> bptestbpcd main: Function ConnectToBPCD(server1) failed: 46
<16>bptestbpcd main: Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.
Host bck1oam (11.5.14.2) is not an authorized server for host server2.
23:14:16.814 [5628] <16> bptestbpcd main: Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.
Host bck1oam (11.5.14.2) is not an authorized server for host server2.
<2>bptestbpcd: server not allowed access
23:14:16.814 [5628] <2> bptestbpcd: server not allowed access
<2>bptestbpcd: EXIT status = 46
23:14:16.814 [5628] <2> bptestbpcd: EXIT status = 46

As suggested in the forums, i've checked the host files in both servers and they're ok. also resolution works.

bp.conf contains the entries in both servers.

[root@server1 bpcd]# netstat -a|grep bpcd
tcp 0 0 0.0.0.0:bpcd 0.0.0.0:* LISTEN
unix 2 [ ACC ] STREAM LISTENING 1401475 /usr/openv/var/vnetd/terminate_bpcd.uds
unix 2 [ ACC ] STREAM LISTENING 1401477 /usr/openv/var/vnetd/bpcd.uds
unix 3 [ ] STREAM CONNECTED 1399505 /tmp/PBXPIPEbpcd
[root@server1 bpcd]# pwd
/usr/openv/netbackup/logs/bpcd
[root@server1 bpcd]# cd ../../
[root@server1 netbackup]# cat bp.conf
SERVER = server1
SERVER = server2
CLIENT_NAME = server1
USE_VXSS = PROHIBITED
VXSS_SERVICE_TYPE = INTEGRITYANDCONFIDENTIALITY
EMMSERVER = server1
HOST_CACHE_TTL = 3600
VXDBMS_NB_DATA = /usr/openv/db/data
ALLOW_MEDIA_OVERWRITE = DBR
ALLOW_MEDIA_OVERWRITE = TAR
ALLOW_MEDIA_OVERWRITE = CPIO
ALLOW_MEDIA_OVERWRITE = ANSI
ALLOW_MEDIA_OVERWRITE = AOS/VS
ALLOW_MEDIA_OVERWRITE = MTF1
ALLOW_MEDIA_OVERWRITE = RS-MTF1
ALLOW_MEDIA_OVERWRITE = BE-MTF1
JOB_PRIORITY = 0 0 90000 90000 90000 90000 85000 85000 80000 80000 80000 80000 75000 75000 70000 70000 50000 50000 45000 0 0 0 0 0
CLIENT_PORT_WINDOW = 0 0
VERBOSE = 5
ENABLE_ROBUST_LOGGING = YES
TRUSTED_MASTER = bckserver2
TELEMETRY_UPLOAD = YES
[root@server1 netbackup]#

Firewall rules are flushed in both servers.

[root@server1 netbackup]# iptables --list
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

[root@server1 netbackup]# bptestnetconn
SERVER = server1
SERVER = server2
------------------------------------------------------------------------
NBU IP_ADDRESS_FAMILY configured to use Remote Addresses: IPv4(yes) IPv6(no)
FL: server1 -> 11.5.14.4 : 0 ms FAST (< 5 sec) [local] SRC: ANY
FL: server2 -> 11.5.11.4 : 2 ms FAST (< 5 sec) SRC: ANY
------------------------------------------------------------------------
RL: 11.5.14.4 -> server1 : 0 ms FAST (< 5 sec)
RL: 11.5.11.4 -> server2 : 0 ms FAST (< 5 sec)
RL: ::1 -> localhost : 0 ms FAST (< 5 sec)
RL: 127.0.0.1 -> localhost : 0 ms FAST (< 5 sec)
------------------------------------------------------------------------
Using configured hostname server1
Displaying all locally configured IP addresses
and if these addresses are found in the DNS

Address Local DNS Entry
------- ----- ----------
11.5.14.4 Yes Yes
11.5.14.2 Yes No - private/autoconfigured
11.5.14.1 Yes No - private/autoconfigured

No PREFERRED_NETWORK directives.
------------------------------------------------------------------------
Total elapsed time: 0 sec
[root@server1 netbackup]#

bpcd log tail:

23:51:39.826 [834] <2> vnet_pbxAcceptSocket: Accepted sock[11] from 11.5.14.4:35196
23:51:39.826 [834] <2> bpcd main: accept sock = 11
23:51:39.827 [7788] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 0 0x0
23:51:39.827 [7788] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 1 0x1
23:51:39.827 [7788] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 2 0x2
23:51:39.827 [7788] <2> ReadKeyfile: keyfile /usr/openv/netbackup/keyfile does not exist

23:51:39.827 [7788] <2> process_requests: offset to GMT -3600
23:51:39.827 [7788] <2> logconnections: BPCD ACCEPT FROM 11.5.14.4.35196 TO 11.5.14.4.1556 fd = 0
23:51:39.827 [7788] <2> process_requests: setup_sockopts complete
23:51:39.827 [7788] <2> vnet_pcache_init_table: [vnet_private.c:212] starting cache size 200 0xc8
23:51:39.828 [7788] <2> vnet_cached_getnameinfo: [vnet_addrinfo.c:1922] found via getnameinfo OUR_HOST=server1 IPS
TR=11.5.14.4
23:51:39.832 [7788] <2> print_arp_info: server1 (11.5.14.4) -- no entry
23:51:39.832 [7788] <2> bpcd peer_hostname: Connection from host server1 (11.5.14.4) port 35196
23:51:39.832 [7788] <2> bpcd valid_server: comparing server1 and server1
23:51:39.832 [7788] <4> bpcd valid_server: hostname comparison succeeded
23:51:39.832 [7788] <2> process_requests: output socket port number = 1
23:51:39.908 [7788] <2> vnet_receive_network_socket: [vnet_vnetd.c:1084] hash_str1 916af2ca655dda79a517dbc1aebc6c74
23:51:39.908 [7788] <2> vnet_receive_network_socket: [vnet_vnetd.c:1085] hash_str2 e90f593d755c9f65d6e0bc06d7692e7d
23:51:39.908 [7788] <2> verify_hashes: [vnet_vnetd.c:1615] hash_str1 916af2ca655dda79a517dbc1aebc6c74
23:51:39.908 [7788] <2> verify_hashes: [vnet_vnetd.c:1617] hash_str2 e90f593d755c9f65d6e0bc06d7692e7d
23:51:39.908 [7788] <2> verify_hashes: [vnet_vnetd.c:1643] hash_str 916af2ca655dda79a517dbc1aebc6c74
23:51:39.908 [7788] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 7 0x7
23:51:39.908 [7788] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 2 0x2
23:51:39.908 [7788] <2> process_requests: Duplicated vnetd socket on stderr
23:51:39.908 [7788] <2> process_requests: <---- NetBackup 7.6 0 ------------initiated
23:51:39.908 [7788] <2> process_requests: VERBOSE = 5
23:51:39.908 [7788] <2> process_requests: Not using VxSS authentication with server1
23:51:39.947 [7788] <2> process_requests: BPCD_GET_VERSION_RQST
23:51:39.947 [7788] <2> process_requests: BPCD_WANT_STATUSMSGS_RQST
23:51:39.947 [7788] <2> process_requests: BPCD_IMMED_CMD_RQST
23:51:39.947 [7788] <2> process_requests: immed cmd = /usr/openv/netbackup/bin/bptm -delete_all_expired 1>/dev/null 2
>/dev/null
23:51:39.947 [7788] <2> sanitary_mb_str: detected /dev/null after redirection
23:51:39.948 [7788] <2> sanitary_mb_str: detected /dev/null after redirection
23:51:39.948 [7788] <2> sanitary_mb_str: String "/usr/openv/netbackup/bin/bptm -delete_all_expired 1>/dev/null 2>/dev
/null" is considered sanitary.
23:51:40.091 [7788] <2> process_requests: completed command
23:51:40.091 [7788] <2> process_requests: immed cmd status = 0x0
23:51:40.091 [7788] <2> process_requests: immed cmd status = 0
23:51:40.091 [7788] <2> process_requests: BPCD_DISCONNECT_RQST
23:51:40.091 [7788] <2> bpcd exit_bpcd: exit status 0 ----------->exiting
23:51:40.092 [834] <2> daemon_check_for_zombies: waited for 1 child processes including: 7788
23:51:42.875 [834] <2> vnet_pbxAcceptSocket: Accepted sock[11] from 11.5.14.4:38106
23:51:42.875 [834] <2> bpcd main: accept sock = 11
23:51:42.876 [7795] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 0 0x0
23:51:42.876 [7795] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 1 0x1
23:51:42.876 [7795] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 2 0x2
23:51:42.876 [7795] <2> ReadKeyfile: keyfile /usr/openv/netbackup/keyfile does not exist

23:51:42.876 [7795] <2> process_requests: offset to GMT -3600
23:51:42.876 [7795] <2> logconnections: BPCD ACCEPT FROM 11.5.14.4.38106 TO 11.5.14.4.1556 fd = 0
23:51:42.876 [7795] <2> process_requests: setup_sockopts complete
23:51:42.876 [7795] <2> vnet_pcache_init_table: [vnet_private.c:212] starting cache size 200 0xc8
23:51:42.877 [7795] <2> vnet_cached_getnameinfo: [vnet_addrinfo.c:1922] found via getnameinfo OUR_HOST=server1 IPS
TR=11.5.14.4
23:51:42.880 [7795] <2> print_arp_info: server1 (11.5.14.4) -- no entry
23:51:42.881 [7795] <2> bpcd peer_hostname: Connection from host server1 (11.5.14.4) port 38106
23:51:42.881 [7795] <2> bpcd valid_server: comparing server1 and server1
23:51:42.881 [7795] <4> bpcd valid_server: hostname comparison succeeded
23:51:42.881 [7795] <2> process_requests: output socket port number = 1
23:51:42.957 [7795] <2> vnet_receive_network_socket: [vnet_vnetd.c:1084] hash_str1 34bbd34f314ba871b4ff8180ce2759ee
23:51:42.957 [7795] <2> vnet_receive_network_socket: [vnet_vnetd.c:1085] hash_str2 0d2a625d74b1ff3ad41fff47144b8d80
23:51:42.957 [7795] <2> verify_hashes: [vnet_vnetd.c:1615] hash_str1 34bbd34f314ba871b4ff8180ce2759ee
23:51:42.957 [7795] <2> verify_hashes: [vnet_vnetd.c:1617] hash_str2 0d2a625d74b1ff3ad41fff47144b8d80
23:51:42.957 [7795] <2> verify_hashes: [vnet_vnetd.c:1643] hash_str 34bbd34f314ba871b4ff8180ce2759ee
23:51:42.957 [7795] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 7 0x7
23:51:42.957 [7795] <2> vnet_check_resilient_socket: [vnet_nbrntd.c:806] the socket is 2 0x2
23:51:42.957 [7795] <2> process_requests: Duplicated vnetd socket on stderr
23:51:42.957 [7795] <2> process_requests: <---- NetBackup 7.6 0 ------------initiated
23:51:42.957 [7795] <2> process_requests: VERBOSE = 5
23:51:42.957 [7795] <2> process_requests: Not using VxSS authentication with server1
23:51:42.996 [7795] <2> process_requests: BPCD_GET_VERSION_RQST
23:51:42.996 [7795] <2> process_requests: BPCD_WANT_STATUSMSGS_RQST
23:51:42.998 [7795] <2> process_requests: BPCD_FORK_CMD_RQST
23:51:42.999 [7795] <2> process_requests: fork cmd = /usr/openv/netbackup/bin/bptm bptm -rptdrv -jobid -1428352904 -j
m
23:51:42.999 [7795] <2> process_requests: filter type = -1
23:51:42.999 [7795] <2> process_requests: backuptime before fork = 0
23:51:42.999 [7795] <2> bpcd exit_bpcd: exit status 0 ----------->exiting
23:51:42.999 [7799] <2> process_requests: Converting args string into an argv
23:51:42.999 [7799] <2> process_requests: Done converting args string into an argv
23:51:42.999 [7799] <2> process_requests: child_args[0] = /usr/openv/netbackup/bin/bptm
23:51:42.999 [7799] <2> process_requests: child_args[1] = bptm
23:51:42.999 [7799] <2> process_requests: child_args[2] = -rptdrv
23:51:42.999 [7799] <2> process_requests: child_args[3] = -jobid
23:51:42.999 [7799] <2> process_requests: child_args[4] = -1428352904
23:51:42.999 [7799] <2> process_requests: child_args[5] = -jm
23:51:42.999 [7799] <2> process_requests: Before execvp of command
23:51:43.000 [834] <2> daemon_check_for_zombies: waited for 1 child processes including: 7795
[root@server1 bpcd]#

Also, bpclntcmd -clear_host_cache tried but did not work.

Please can you help me?

thank you in advance!

RiaanBadenhorst · ‎04-07-2015

Hi,

Make sure server1 is in the server list (bp.conf) of server2, and vice versa. And make sure that you've tested the comms/resolution both ways.

From server1

bptestbpcd -host server2

bpclntcmd -hn server2

bpcltncmd -ip "IP of server2"

From server2

bptestbpcd -host server1

bpclntcmd -hn server1

bpcltncmd -ip "IP of server1"

Error from your log shows host is not allowed access.

<16>bptestbpcd main: Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.
Host bck1oam (11.5.14.2) is not an authorized server for host server2.
23:14:16.814 [5628] <16> bptestbpcd main: Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.
Host bck1oam (11.5.14.2) is not an authorized server for host server2.
<2>bptestbpcd: server not allowed access
23:14:16.814 [5628] <2> bptestbpcd: server not allowed access
<2>bptestbpcd: EXIT status = 46
23:14:16.814 [5628] <2> bptestbpcd: EXIT status = 46

View solution in original post

RiaanBadenhorst · ‎04-07-2015

Hi,

Make sure server1 is in the server list (bp.conf) of server2, and vice versa. And make sure that you've tested the comms/resolution both ways.

From server1

bptestbpcd -host server2

bpclntcmd -hn server2

bpcltncmd -ip "IP of server2"

From server2

bptestbpcd -host server1

bpclntcmd -hn server1

bpcltncmd -ip "IP of server1"

Error from your log shows host is not allowed access.

<16>bptestbpcd main: Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.
Host bck1oam (11.5.14.2) is not an authorized server for host server2.
23:14:16.814 [5628] <16> bptestbpcd main: Request from host bck1oam (11.5.14.2) to host server2 (11.5.11.4) is not allowed access.
Host bck1oam (11.5.14.2) is not an authorized server for host server2.
<2>bptestbpcd: server not allowed access
23:14:16.814 [5628] <2> bptestbpcd: server not allowed access
<2>bptestbpcd: EXIT status = 46
23:14:16.814 [5628] <2> bptestbpcd: EXIT status = 46

Marianne · ‎04-07-2015

As per Riaan's post - you are missing a SERVER entry for bck1oam in bp.conf on server2 : "Host bck1oam (11.5.14.2) is not an authorized server for host server2." Restart NBU on server2 after adding SERVER entry.

Handy NetBackup Links

VOX

Master server reject connections-Adding trusted server