11-03-2012 08:12 AM
Hi all,
I have an environment consist of cluster master server windows 2008 and 5 media servers and 2 5020 appliances.I installed agent on the linux redhat client and and i added all server list to bp.conf and etc/hosts in the client and i added the client name in the master server host file.when i'm trying to ping from the client every thing working fine,but i couldnot do this from the master to client.then when i configured the policy the jobs failed with status
Exit Status: 48 (client hostname could not be found)
then I ran command bptestbpcd -client from master with -debug i got this result:
C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -client BEC13-ENC2-2
-debug
16:09:32.930 [3444.5668] <2> bptestbpcd: VERBOSE = 0
16:09:32.946 [3444.5668] <2> copy_preferred_network_list: ../../libvlibs/nbconf.
c.1610: pseudo PN for : bec12-encx-v37
16:09:32.946 [3444.5668] <2> copy_preferred_network_list: ../../libvlibs/nbconf.
c.1610: pseudo PN for : bec12-encx-v37
16:09:32.961 [3444.5668] <2> copy_preferred_network_list: ../../libvlibs/nbconf.
c.1610: pseudo PN for : bec12-encx-v37
16:10:02.963 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:10:14.964 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:10:14.964 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:10:45.965 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:10:57.966 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:10:57.966 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:11:08.982 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:11:38.984 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:11:53.984 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:11:57.985 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:12:05.001 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:12:35.002 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:12:43.002 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:12:46.003 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:13:07.004 [3444.5668] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_
ERROR returned 10060 0x274c
16:13:07.004 [3444.5668] <16> connect_to_service: connect failed STATUS (18) CON
NECT_FAILED
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed o
ut.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd VIA pbx
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed o
ut.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd VIA vnetd
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed o
ut.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd
16:13:07.004 [3444.5668] <8> vnet_connect_to_bpcd: [vnet_connect.c:279] connect_
to_service() failed 18 0x12
16:13:07.004 [3444.5668] <2> local_bpcr_connect: Can't connect to client BEC13-E
NC2-2
16:13:07.004 [3444.5668] <2> ConnectToBPCD: bpcd_connect_and_verify(BEC13-ENC2-2
, BEC13-ENC2-2) failed: 25
<16>bptestbpcd main: Function ConnectToBPCD(BEC13-ENC2-2) failed: 25
16:13:07.004 [3444.5668] <16> bptestbpcd main: Function ConnectToBPCD(BEC13-ENC2
-2) failed: 25
<2>bptestbpcd: cannot connect on socket
16:13:07.004 [3444.5668] <2> bptestbpcd: cannot connect on socket
<2>bptestbpcd: EXIT status = 25
16:13:07.004 [3444.5668] <2> bptestbpcd: EXIT status = 25
cannot connect on socket
C:\Program Files\Veritas\NetBackup\bin\admincmd>
please advice what is missed???
Regards,
Solved! Go to Solution.
11-04-2012 10:15 AM
Is status 48 now resolved?
For some or other reason I got distracted from original status 48.
'Client hostname not found' is resolved by adding client IP and hostname to DNS or to master and media server's hosts files as well as double-checking policy for typo errors as far as Client hostname is concerned.
After changes to hosts files (or DNS) were made, refresh NBU host cache on master and media server:
bpclntcmd -clear_host_cache
Status 25 is a separate issue and points to port connectivity.
You have selected NBU 7.5 in your opening post.
If this is correct, you ONLY need port 1556 between master and client as well as between media server and client.
Check that bpcd and vnetd processes as well as pbx processes on the client are running.
This points to problem on the client:
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed out.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd VIA pbx
Please confirm the NBU version on the client as well as processes running on the client.
Do the following from master and media server:
bpstestbpcd -client <client-name> -debug -verbose
11-03-2012 12:53 PM
Ensure that iptables on Linux client is disabled.
If there is a firewall between client and server(s), ensure that port 1556 is open between client and all media servers that are allowed to backup this client.
11-04-2012 09:11 AM
iptables on clients were disabled using:
etc/init.d/iptables stop
but the problem still exist.
symantec support asked me to do the following:
CLIENT CONNECTIVITY
After installation of netbackup and license keys on master and or media servers and installing client software we have to make sure we are able to communicate with clients on ports 1556(pbx),13724(vnetd), 13782(bpcd).
(bptestbpcd –client not responding)
(Ping client name from master or media not responding)
[root@BEC13-ENC1-1 ~]# netstat -a | more
netstat -at
tcp 0 0 *:bpcd *:* LISTEN
netstat –au
[root@BEC13-ENC1-1 ~]# netstat -lt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:sunrpc *:* LISTEN
tcp 0 0 localhost:38961 *:* LISTEN
tcp 0 0 *:veritas_pbx *:* LISTEN
tcp 0 0 *:57364 *:* LISTEN
tcp 0 0 localhost:arbortext-lm *:* LISTEN
tcp 0 0 *:bpcd *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 localhost:ipp *:* LISTEN
tcp 0 0 localhost:smtp *:* LISTEN
tcp 0 0 *:vnetd *:* LISTEN
tcp 0 0 *:amqp *:* LISTEN
tcp 0 0 *:sunrpc *:* LISTEN
tcp 0 0 *:33392 *:* LISTEN
tcp 0 0 *:veritas_pbx *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 localhost:ipp *:* LISTEN
tcp 0 0 localhost:smtp *:* LISTEN
[root@BEC13-ENC1-1 ~]#
These ports should be opened bi-directional between master and media server to the client. Make note of the outputs from netstat (alias server comes in as is very important). The aliases will usually need to be in hosts or DNS and bp.conf or server lists.
result :I'm waiting the customer to open the ports bi-directional
*******************************
1. ensured connectivity via ports 1556 and 13782 and 13724 from master server
telnet <media server> 13782 (Leave this connection and go to the client and run netstat commands)
**********************************
result:the connection opened with all ports from master server normally
on media server
for Unix - netstat -a |grep bpcd (look at server coming in is it the shortname, is it the FQDN or an alias that is not defined in the client bp.conf (Unix) or the server list (for windows)
for Windows - netstat -a |findstr bpcd (look at server coming in is it the shortname, is it the FQDN or an alias that is not defined in the client bp.conf (Unix) or the server list (for windows)
********************************
result :all hosts were added to bp.conf of the client
2. Same connectivity is necessary from the media server
If your are able to connect to the ports but still not able to connect there is likely a resolution issue sh check nslookup and hosts file entries.
Can also run a handy command to see if you are able to connect to the client from master or media server run:
/usr/openv/netbackup/bin/admincmd/bptestbpcd -client <clientname>
No response after time getting status 48Exit Status: 48 (client hostname could not be found)
Regards,
11-04-2012 09:20 AM
one more thing:
I have 4 identical linux servers after testing all previous commands i tryed to add one of these servers in the client attribute inside the master server proporties and i added the the host name then i clicked double click on client it was connected then i added the media servers to the server list and then run backup i recieved successful backup.then i did the same thing in all other servers but it didnt work!!!!!!!
got Status Code: 25 Cannot connect on socket
11-04-2012 10:15 AM
Is status 48 now resolved?
For some or other reason I got distracted from original status 48.
'Client hostname not found' is resolved by adding client IP and hostname to DNS or to master and media server's hosts files as well as double-checking policy for typo errors as far as Client hostname is concerned.
After changes to hosts files (or DNS) were made, refresh NBU host cache on master and media server:
bpclntcmd -clear_host_cache
Status 25 is a separate issue and points to port connectivity.
You have selected NBU 7.5 in your opening post.
If this is correct, you ONLY need port 1556 between master and client as well as between media server and client.
Check that bpcd and vnetd processes as well as pbx processes on the client are running.
This points to problem on the client:
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed out.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd VIA pbx
Please confirm the NBU version on the client as well as processes running on the client.
Do the following from master and media server:
bpstestbpcd -client <client-name> -debug -verbose
11-04-2012 10:34 AM
first:AfterI added the ip address of the client to the host file in the masre server i got 48 status. then after following the above steps i got staus 25.
secondry : i restart all services in the master server, so do you thing ths will not refresh NBU host cash?
third:the master server version is 7.5 and the client also 7.5 . I will ask the customer to open this port 1556 between the master and the client.
forth:
Check that bpcd and vnetd processes as well as pbx processes on the client are running.
when i run /etc/services i got nothing =========
how to check these services other way?
fifth:
bpstestbpcd -client <client-name> -debug -verbose
i ran this command from the master and media server and got the status 25 error.I will run this command again tommorow.
Regards
11-04-2012 10:01 PM
I refresh NBU host cache on master and media server then run
C:\Program Files\Veritas\NetBackup\bin>bptestbpcd -client BEC13-ENC1-2 -debug -v
erbose
'bptestbpcd' is not recognized as an internal or external command,
operable program or batch file.
C:\Program Files\Veritas\NetBackup\bin>cd admincmd
C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -client BEC13-ENC1-2
-debug -verbose
08:56:02.224 [3412.752] <2> bptestbpcd: VERBOSE = 0
08:56:02.224 [3412.752] <2> read_client: dname=., offline=0, online_at=0 offline
_at=0
08:56:02.224 [3412.752] <2> read_client: dname=.., offline=0, online_at=0 offlin
e_at=0
08:56:02.224 [3412.752] <2> read_client: dname=CO_0, offline=0, online_at=0 offl
ine_at=0
08:56:02.224 [3412.752] <2> read_client: dname=host_info, offline=0, online_at=0
offline_at=0
08:56:02.224 [3412.752] <2> read_client: dname=OA_2147483647, offline=0, online_
at=2147483647 offline_at=0
08:56:02.224 [3412.752] <2> db_freeEXDB_INFO: ?
08:56:02.240 [3412.752] <2> copy_preferred_network_list: ../../libvlibs/nbconf.c
.1610: pseudo PN for : bec12-encx-v37
08:56:02.255 [3412.752] <2> copy_preferred_network_list: ../../libvlibs/nbconf.c
.1610: pseudo PN for : bec12-encx-v37
08:56:02.255 [3412.752] <2> copy_preferred_network_list: ../../libvlibs/nbconf.c
.1610: pseudo PN for : bec12-encx-v37
08:56:02.271 [3412.752] <2> vnet_pbxConnect: ../../libvlibs/vnet_pbx.c.666: pbxS
etAddrEx/pbxConnectEx return error 0:The operation completed successfully.
08:56:02.271 [3412.752] <8> do_pbx_service: [vnet_connect.c:2034] vnet_pbxConnec
t() failed 18 0x12
08:56:02.271 [3412.752] <8> do_pbx_service: [vnet_connect.c:2035] save_errno 0 0
x0
08:56:02.271 [3412.752] <8> do_pbx_service: [vnet_connect.c:2036] use_vnetd 0 0x
0
08:56:02.271 [3412.752] <8> do_pbx_service: [vnet_connect.c:2037] cr->vcr_servic
e bpcd
08:56:02.271 [3412.752] <8> async_connect: [vnet_connect.c:1630] do_service fail
ed 18 0x12
08:56:03.302 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:04.334 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:06.334 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:07.334 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:09.334 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:10.365 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:14.381 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:15.397 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:23.428 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:24.444 [3412.752] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_E
RROR returned 10061 0x274d
08:56:24.444 [3412.752] <16> connect_to_service: connect failed STATUS (18) CONN
ECT_FAILED
status: FAILED, (24) BAD_VERSION; system: (10036) A blocking operation i
s currently executing. ; FROM 172.20.124.11 TO BEC13-ENC1-2 172.20.165.2 bpcd VI
A pbx
status: FAILED, (42) CONNECT_REFUSED; system: (10061) No connection coul
d be made because the target machine actively refused it. ; FROM 172.20.124.11 T
O BEC13-ENC1-2 172.20.165.2 bpcd VIA vnetd
status: FAILED, (42) CONNECT_REFUSED; system: (10061) No connection coul
d be made because the target machine actively refused it. ; FROM 172.20.124.11 T
O BEC13-ENC1-2 172.20.165.2 bpcd
08:56:24.444 [3412.752] <8> vnet_connect_to_bpcd: [vnet_connect.c:279] connect_t
o_service() failed 18 0x12
08:56:24.444 [3412.752] <2> local_bpcr_connect: Can't connect to client BEC13-EN
C1-2
08:56:24.444 [3412.752] <2> ConnectToBPCD: bpcd_connect_and_verify(BEC13-ENC1-2,
BEC13-ENC1-2) failed: 25
<16>bptestbpcd main: Function ConnectToBPCD(BEC13-ENC1-2) failed: 25
08:56:24.460 [3412.752] <16> bptestbpcd main: Function ConnectToBPCD(BEC13-ENC1-
2) failed: 25
<2>bptestbpcd: cannot connect on socket
08:56:24.460 [3412.752] <2> bptestbpcd: cannot connect on socket
<2>bptestbpcd: EXIT status = 25
08:56:24.460 [3412.752] <2> bptestbpcd: EXIT status = 25
cannot connect on socket
C:\Program Files\Veritas\NetBackup\bin\admincmd>
11-04-2012 11:31 PM
Hi
Ensure the Firewall is stopped in the master server and client machine
In client
/usr/openv/netbackup/bin/bpcd -stanalone
11-04-2012 11:53 PM
Please show us processes running on the client.
We need to see :
/usr/openv/netbackup/bin/bpcd -standalone
/usr/openv/netbackup/bin/vnetd -standalone
/opt/VRTSpbx/bin/pbx_exchange
11-05-2012 12:08 AM
[root@BEC13-ENC1-1 bin]# bpcd -standalone
-bash: bpcd: command not found
[root@BEC13-ENC1-1 bin]# vnetd -standalone
-bash: vnetd: command not found
[root@BEC13-ENC1-1 bin]# pbx_exchange
-bash: pbx_exchange: command not found
[root@BEC13-ENC1-1 bin]#
11-05-2012 01:04 AM
Check output of 'ps -ef' and look for these processes.
They will be listed with full pathnames.
Output on our lab Linux machine:
11-05-2012 01:04 AM
11-05-2012 01:15 AM
NB Processes
------------
root 5674 5633 0 16:40 ? 00:00:05 /usr/openv/java/jre/bin/java -Dvrts.NBJAVA_CONF=/usr/openv/java/nbj.conf -showversion -Dvrts.common.server.LCCTYPE="en_US.UTF-8" -Dvrts.common.server.LOG_CMDLINES=false -Djava.library.path=/usr/openv/lib -Dvrtsat.datadir=/usr/openv/var/vxss/at -Dvrtsat.home=/usr/openv/netbackup/sec/at -Dvrts.common.utilities.DEBUG_PROPERTIES=/usr/openv/java/Debug.properties -Dvrts.nbe.HOSTNAME= -Dvrts.nbe.INST_REL=750000 -Dvrts.nbe.BUILDNO=20120207 -Djava.util.prefs.PreferencesFactory=vrts.common.launch.PropertiesPreferencesFactory -Xms36M -Xmx256M vrts.nbe.UserConsole
root 5766 5725 0 16:46 ? 00:00:12 /usr/openv/java/jre/bin/java -Dvrts.NBJAVA_CONF=/usr/openv/java/nbj.conf -showversion -Dvrts.common.server.LCCTYPE="en_US.UTF-8" -Dvrts.common.server.LOG_CMDLINES=false -Djava.library.path=/usr/openv/lib -Dvrtsat.datadir=/usr/openv/var/vxss/at -Dvrtsat.home=/usr/openv/netbackup/sec/at -Dvrts.common.utilities.DEBUG_PROPERTIES=/usr/openv/java/Debug.properties -Dvrts.nbe.HOSTNAME= -Dvrts.nbe.INST_REL=750000 -Dvrts.nbe.BUILDNO=20120207 -Djava.util.prefs.PreferencesFactory=vrts.common.launch.PropertiesPreferencesFactory -Xms36M -Xmx256M vrts.nbe.UserConsole
root 27231 1 0 Nov04 ? 00:00:01 /usr/openv/netbackup/bin/vnetd -standalone
root 27236 1 0 Nov04 ? 00:00:01 /usr/openv/netbackup/bin/bpcd -standalone
11-05-2012 01:33 AM
Which command did you use to list processes? I don't see pbx.
Please try the command in my example? # ps -ef | egrep 'vnetd|bpcd|pbx'
It seems there is still something that is blocking port connecttivity.
Try the following from the master and media server:
telnet BEC13-ENC1-2 13724
If this fails - you need to find out what other software on client is blocking port connectivity or else if there is a firewall between servers and client.
We can see that connection to ALL of these ports is refused by the client:
CONNECT_FAILED
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed out.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd VIA pbx
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed out.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd VIA vnetd
status: FAILED, (42) CONNECT_REFUSED; system: (10060) Connection timed out.; FROM 172.20.124.11 TO BEC13-ENC2-2 172.20.165.4 bpcd
11-05-2012 01:55 AM
I used bpps -x
[root@BEC13-ENC1-1 ~]# ps -ef | egrep 'vnetd|bpcd|pbx'
root 8950 8930 0 20:55 pts/1 00:00:00 egrep vnetd|bpcd|pbx
root 25033 1 0 Nov03 ? 00:00:00 /opt/VRTSpbx/bin/pbx_exchange
root 27231 1 0 Nov04 ? 00:00:01 /usr/openv/netbackup/bin/vnetd -standalone
root 27236 1 0 Nov04 ? 00:00:01 /usr/openv/netbackup/bin/bpcd -standalone
[root@BEC13-ENC1-1 ~]#
telnet BEC13-ENC1-2 13724 failed .so i think it is ports issue .I will work on openning all needed ports the will test this again,
Thanks for you all ,Regards,
11-05-2012 02:22 AM
For NBU 7.5, only pbx (port 1556) is needed.
Port needs to be opened bi-directional between master and client as well as media server and client.
Please also check for TCP wrappers on Linix client: http://www.symantec.com/docs/TECH56080
10-26-2016 08:11 AM
Thanks Marianne,
I had same problem, and before do bpclntcmd -clear_host_cache, resolve my problem !
[root@XXXX ~]# bptestbpcd -client joo
<16>bptestbpcd main: Function ConnectToBPCD(joo) failed: 48
<16>bptestbpcd main: client hostname could not be found
client hostname could not be found