cancel
Showing results for 
Search instead for 
Did you mean: 

Symantec PBX service stopped

smwoodcrafts
Level 6
I am having a problem in that a backup that had been running fine started failing with a 21 error. After looking into it and comparing I found that port 1556 was not listening on the Client and Symantec PBX service was not running and can't be restarted. Is there something I can look at here? We already restarted the server and PBX didn't start.  Here are the particulars:

Master and Media servers are running Windows 2003 with NetBackup 6.5.3.1
Client in DMZ Windows 2008 standard 64 bit and client is 6.5.2a 2008 64 bit client.

Please help.

Dan Seymour
smwoodcrafts
1 ACCEPTED SOLUTION

Accepted Solutions

smwoodcrafts
Level 6
Finally got this working. I opened a ticket and we were getting no where. Some one in backline suggested we go deeper in the Windows Firewall and stop all three profiles and try the backup. I checked the configs for this and netbackup was in as an exception. I did as instructed and the job ran. I decided to turn everything back on and try again and the job ran again. It appears that, when the box went through Windows update, it screwed with the firewall somehow. When I bounced it, it went back to normal. Now it's running fine, without changing one configuration.

Thanks all. Hopefully this helps someone else out.

smwoodcrafts

View solution in original post

18 REPLIES 18

Christoph_Linde
Level 5
Employee
Hi,

look into the application event log for the Windows Server. You should find reasons, why the PBX service is stopping.

Neverthless, I suggest updating the client anyways. at least 6.5.3.1.

Also consider updating the master server to 6.5.5 (and the client also)

smwoodcrafts
Level 6
As an update, I just uninstalled the client, rebooted the system and then reinstalled the client. The PBX still didn't start. What could stop this service from starting? looking in the application event log, there is no event ID I get a 0 there

smwoodcrafts
Level 6
Is there a Windows 2008 64bit client as in 6.5.2, or is it covered with the regular 64bit client?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

PBX does not need to run on a client - only on master and media servers.
Verify that vnetd (1724) is still open (bi-directional) between master & client and media server and client.
Test with the following from master and media server:
bptestbpcd -verbose -client <client-name>
Ensure you have bpcd log directory on client before running the tests.

Will_Restore
Level 6
runs on the Master server.


VERITAS NetBackup (tm) 6.0 Port Usage Guide for Windows and UNIX Platforms http://seer.entsupport.symantec.com/docs/281623.htm

Christoph_Linde
Level 5
Employee
You can find the client support information here:

ftp://exftpp.symantec.com/pub/support/products/NetBackup_Enterprise_Server/325328.pdf

For the Windows 2008 x64 Client: It is supported since 6.5.2, yes.

But the guys are correct for a simple client. BPX is only used on Master/Media OR if you use VxSS.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Windows 2008 64bit is covered with the regular 6.5.2 64bit client, except if W2008 R2 - only supported as from 6.5.5. PLEASE upgrade to something later than 6.5.2 - it was probably the most problematic 6.5.x version...

smwoodcrafts
Level 6
Ok, then I need to ask a stupid question. This was working, going through a firewall, for a while. It then stopped working. I can't find any reason why it stopped. I looked at what was running on a similar box on the same domain. Both having everything appear the same exept, on the box that is still connecting Symantec PBX is running. When I do netstat to check the ports 13724, 13782, 13783 and 1556 is listening on the good box 1556 isn't listening on the box that won't connect. When I connect through the host properties on the master server. I see the port connections between the master and client on 13724, 6 of them in the WAIT_STATE. On the box that won't connect, I see 2 port connections ESTABLISHED between the master and client.  What am I missing here? This is driving me crazy.

smwoodcrafts

Christoph_Linde
Level 5
Employee
pls try from master:

bptestbpcd -client <clientname> -verbose -debug

It is located in netbackup\bin\admincmd (i guess)

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Maybe changes on firewall or Windows updates on client? Have you asked Firewall admins to monitor firewall traffic?
Maybe 'someone' was 'playing' with connect options on master server?
Have you tried bptestbpcd yet?
If so, what was the result and what's in bpcd log on client?

smwoodcrafts
Level 6
Ok I tried bptestbpcd on master and media server. The results are below. I didn't use the -debug argument. What exactly does that do? I did check the firewall first. This server is part of a group if there was a problem at the firewall, all of them would be failing. Please take a look and see if you can find a problem. I'm somewhat new to the troubleshooting here and I really appreciate any help you can give me.

14:47:25.087 [3592.3456] <2> bpcd main: offset to GMT 0
14:47:25.087 [3592.3456] <2> bpcd main: Got socket for input 536
14:47:25.087 [3592.3456] <2> logconnections: BPCD ACCEPT FROM 10.10.4.134.4185 TO 10.10.116.27.13724
14:47:25.087 [3592.3456] <2> bpcd main: setup_sockopts complete
14:47:25.087 [3592.3456] <2> bpcd peer_hostname: Connection from host lonbackup00 (10.10.4.134) port 4185
14:47:25.087 [3592.3456] <2> bpcd valid_server: comparing lonbackup00 and lonbackup00
14:47:25.087 [3592.3456] <4> bpcd valid_server: hostname comparison succeeded
14:47:25.275 [3592.3456] <2> bpcd main: output socket port number = 1
14:47:46.772 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:48:08.800 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:48:31.810 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:48:35.211 [3660.3532] <2> bpcd main: offset to GMT 0
14:48:35.211 [3660.3532] <2> bpcd main: Got socket for input 544
14:48:35.211 [3660.3532] <2> logconnections: BPCD ACCEPT FROM 10.10.4.134.4593 TO 10.10.116.27.13724
14:48:35.211 [3660.3532] <2> bpcd main: setup_sockopts complete
14:48:35.211 [3660.3532] <2> bpcd peer_hostname: Connection from host lonbackup00 (10.10.4.134) port 4593
14:48:35.211 [3660.3532] <2> bpcd valid_server: comparing lonbackup00 and lonbackup00
14:48:35.211 [3660.3532] <4> bpcd valid_server: hostname comparison succeeded
14:48:35.398 [3660.3532] <2> bpcd main: output socket port number = 1
14:48:56.631 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:48:56.833 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:49:18.643 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:49:25.834 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:49:41.653 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:49:42.839 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10061 0x0000274d
14:49:42.839 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.261: timo: 32 0x00000020
14:49:42.839 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.263: connect_refused_tries: 16 0x00000010
14:49:42.839 [3592.3456] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.287: Function failed: 10 0x0000000a
14:49:42.839 [3592.3456] <2> vnet_bind_and_connect: ..\libvlibs\vnet_bind.c.175: Function failed: 10 0x0000000a
14:49:42.839 [3592.3456] <2> vnet_connect_by_vnetd: ..\libvlibs\vnet_vnetd.c.1416: status: 10 0x0000000a
14:49:42.839 [3592.3456] <2> get_vnetd_forward_socket: vnet_connect_by_vnetd failed: 10
14:49:42.839 [3592.3456] <16> bpcd main: get_vnetd_forward_socket failed: 21
14:50:06.692 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:50:35.693 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:50:52.697 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10061 0x0000274d
14:50:52.697 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.261: timo: 32 0x00000020
14:50:52.697 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.263: connect_refused_tries: 16 0x00000010
14:50:52.697 [3660.3532] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.287: Function failed: 10 0x0000000a
14:50:52.697 [3660.3532] <2> vnet_bind_and_connect: ..\libvlibs\vnet_bind.c.175: Function failed: 10 0x0000000a
14:50:52.697 [3660.3532] <2> vnet_connect_by_vnetd: ..\libvlibs\vnet_vnetd.c.1416: status: 10 0x0000000a
14:50:52.697 [3660.3532] <2> get_vnetd_forward_socket: vnet_connect_by_vnetd failed: 10
14:50:52.697 [3660.3532] <16> bpcd main: get_vnetd_forward_socket failed: 21
14:52:50.527 [4052.3792] <2> bpcd main: offset to GMT 0
14:52:50.527 [4052.3792] <2> bpcd main: Got socket for input 536
14:52:50.527 [4052.3792] <2> logconnections: BPCD ACCEPT FROM 10.10.4.135.2180 TO 10.10.116.27.13724
14:52:50.527 [4052.3792] <2> bpcd main: setup_sockopts complete
14:52:50.527 [4052.3792] <2> bpcd peer_hostname: Connection from host lonbackup02 (10.10.4.135) port 2180
14:52:50.527 [4052.3792] <2> bpcd valid_server: comparing lonbackup00 and lonbackup02
14:52:50.543 [4052.3792] <2> bpcd valid_server: comparing lonbackup02 and lonbackup02
14:52:50.543 [4052.3792] <4> bpcd valid_server: hostname comparison succeeded
14:52:50.715 [4052.3792] <2> bpcd main: output socket port number = 1
14:53:12.118 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:53:34.130 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:53:57.141 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:54:22.148 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:54:51.150 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:55:08.170 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10061 0x0000274d
14:55:08.170 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.261: timo: 32 0x00000020
14:55:08.170 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.263: connect_refused_tries: 16 0x00000010
14:55:08.170 [4052.3792] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.287: Function failed: 10 0x0000000a
14:55:08.170 [4052.3792] <2> vnet_bind_and_connect: ..\libvlibs\vnet_bind.c.175: Function failed: 10 0x0000000a
14:55:08.170 [4052.3792] <2> vnet_connect_by_vnetd: ..\libvlibs\vnet_vnetd.c.1416: status: 10 0x0000000a
14:55:08.170 [4052.3792] <2> get_vnetd_forward_socket: vnet_connect_by_vnetd failed: 10
14:55:08.170 [4052.3792] <16> bpcd main: get_vnetd_forward_socket failed: 21
14:56:18.091 [2696.2496] <2> bpcd main: offset to GMT 0
14:56:18.091 [2696.2496] <2> bpcd main: Got socket for input 536
14:56:18.091 [2696.2496] <2> logconnections: BPCD ACCEPT FROM 10.10.4.134.3413 TO 10.10.116.27.13724
14:56:18.091 [2696.2496] <2> bpcd main: setup_sockopts complete
14:56:18.091 [2696.2496] <2> bpcd peer_hostname: Connection from host lonbackup00 (10.10.4.134) port 3413
14:56:18.091 [2696.2496] <2> bpcd valid_server: comparing lonbackup00 and lonbackup00
14:56:18.091 [2696.2496] <4> bpcd valid_server: hostname comparison succeeded
14:56:18.278 [2696.2496] <2> bpcd main: output socket port number = 1
14:56:39.494 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:57:01.522 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:57:24.533 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:57:49.556 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:58:18.557 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10060 0x0000274c
14:58:35.577 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.222: save_errno: 10061 0x0000274d
14:58:35.577 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.261: timo: 32 0x00000020
14:58:35.577 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.263: connect_refused_tries: 16 0x00000010
14:58:35.577 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.287: Function failed: 10 0x0000000a
14:58:35.577 [2696.2496] <2> vnet_bind_and_connect: ..\libvlibs\vnet_bind.c.175: Function failed: 10 0x0000000a
14:58:35.577 [2696.2496] <2> vnet_connect_by_vnetd: ..\libvlibs\vnet_vnetd.c.1416: status: 10 0x0000000a
14:58:35.577 [2696.2496] <2> get_vnetd_forward_socket: vnet_connect_by_vnetd failed: 10
14:58:35.577 [2696.2496] <16> bpcd main: get_vnetd_forward_socket failed: 21

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Any chance that someone has changed connect options for this client on the master server?
Check Host Properties -> Master -> Client Attributes -> Client
Check Connect Options for this client and compare settings with other working clients.

Also have a look at this TechNote: http://seer.entsupport.symantec.com/docs/281801.htm

Will_Restore
Level 6

SOLUTION:
One cause of this issue is incorrectly setting the subnet mask on the client.  To eliminate this as a potential cause of the issue, inspect the subnet mask on the client and ensure that it is set correctly.  To view the subnet mask, run the following command from a command prompt:

ipconfig /all

http://seer.entsupport.symantec.com/docs/296335.htm

smwoodcrafts
Level 6

There is somehing wierd going on. I checked what was suggested in host properties, nothing. I then was checking the IP setting against another box in the domain. On the box I am having problems with, I saw no DNS servers listed and I did on the other box. I asked the owner to check and he looked at the configs and saw DNS entries. I then went back and the DNS entries werenow showing up. Don't know what's going on. I opened a ticket with Symantec and went through everything he told me to do. Sent him the log files and he couldn't find anything, or suggested the same as you guys did.

smwoodcrafts

Dennis_Strom
Level 6
Verify the Client is communicating properly:

    * bpclncmd -ip  <ip address>  --> from both client and server
    * bpclntcmd -hn <hostname>  --> from both client and server
    * bpclntcmd -pn    -->  from client only
     run bpcoverage -c clientname to verify connectivity.
     run nslookup hostname and nslookup IP adress to verify dns is working.
     insure the media servers name is in the client bp.conf
     telnet to client to verify netbackup client telnet clientname bpcd
     You can also ftp a large file from the client to the
        media server to verify connectivity and speed.

http://support.veritas.com/docs/277901
How to test client connections using the bptestbpcd command with Veritas NetBack
up (tm) 6.0.

check hosts files on both machines (Windows has a local hosts file)  if your having problems with DNS you may want to use the local hosts files.


Karthikeyan_Sun
Level 6
Did you check Windows Firewall Port Blockings.. 

Check whether 13782 opened on Windows Firewall, if not  add that port in port block exception list.

Christoph_Linde
Level 5
Employee
14:58:35.577 [2696.2496] <2> vnet_bind_and_connect_extra: ..\libvlibs\vnet_bind.c.263: connect_refused_tries: 16 0x00000010

If you try from the same servers as the bptestbpcd the following command:

telnet <clientname> 13724 ... the result should be a connection refused message, right?

So somewhat blocks your traffic.


smwoodcrafts
Level 6
Finally got this working. I opened a ticket and we were getting no where. Some one in backline suggested we go deeper in the Windows Firewall and stop all three profiles and try the backup. I checked the configs for this and netbackup was in as an exception. I did as instructed and the job ran. I decided to turn everything back on and try again and the job ran again. It appears that, when the box went through Windows update, it screwed with the firewall somehow. When I bounced it, it went back to normal. Now it's running fine, without changing one configuration.

Thanks all. Hopefully this helps someone else out.

smwoodcrafts