Firewall Client Configuration: Unable to configure client on Master

[grosstech]

Hi

The environment consists of the following:

7.0.1 Master on Windows Server 2008 R2

7.0.1 Media Server on SPARC Solaris 10

7.0.1 Client on Windows Server 2008 R2

Client is a new install.

Files used for host name resolution.

Firewall is open for vnetd communication ONLY.

bpclntcmd's working properly ie. -hn and -ip from master, media, client.

ping does not work due to firewall

I have been unable to make bptestbpcd connect using vnetd. It will use non-reserved ports from the initiating side (Master or Media) when attempting to connect to client regardless of connect_options switch.

telnet from the client to Master or media vnetd appears to work ie. a blank screen appears with nothing. In reverse from the Media server receive 'telnet: Unable to connect to remote host: Connection timed out'

I believe that this was working similar to the Windows last Friday.  Windows client required booting because it was unable to telnet to Master or Media server after install.

vnetd appears to be listening on Client.

Master Server Properties -> Firewall -> Default Connect Options: BPCD connect back: VNETD port

                                                                                                        Ports: Reserved port

                                                                                                        Daemon connect port: Automatic

which I believe are defaults. I have modified in attempts to get this to work.

                                                               -> Hosts -> Attributes for selected hosts are both blank although again I have modified. I can input the new client and configure with VNETD port on bpcd callback VNETD only for Daemon connection port.

When going to Host Properties-> Clients and attempting to add new client I receive a cannot connect on socket.

Enabled bpcd logging on client yesterday which looks nothing like posts which I have seen:

$Header 65543,51216,112,1331068607,21600,<client name removed>
0,51216,137,112,1,1331068607812,2880,2460,0:,34:switched to a new logging callback,16:log_set_callback,1
2,51216,137,112,2,1331068607828,2880,2460,0:,0:,0:,0,(74|)
2,51216,137,112,3,1331068607828,2880,2460,0:,0:,0:,0,(13|A10:ProxyV9SVC|)
2,51216,137,112,4,1331068607828,2880,2460,0:,0:,0:,0,(74|)
0,51216,137,112,5,1331068607843,2880,2460,0:,56:endpointvalue is : pbxiop://1556:ProxyV9SVC(Orb.cpp:630),9:Orb::init,1
0,51216,137,112,6,1331068607843,2880,2460,0:,695:initializing ORB ProxyV9SVC with: ProxyV9SVC -ORBSvcConfDirective "-ORBDottedDecimalAddresses 0" -ORBSvcConfDirective "static PBXIOP_Factory '-enable_keepalive'" -ORBSvcConfDirective "static EndpointSelectorFactory ''" -ORBSvcConfDirective "static Resource_Factory '-ORBProtocolFactory PBXIOP_Factory'" -ORBSvcConfDirective "static Resource_Factory '-ORBProtocolFactory IIOP_Factory'" -ORBSvcConfDirective "static PBXIOP_Evaluator_Factory '-orb ProxyV9SVC'" -ORBSvcConfDirective "static Resource_Factory '-ORBConnectionCacheMax 1024 '" -ORBEndpoint pbxiop://1556:ProxyV9SVC -ORBSvcConf nul -ORBSvcConfDirective "static Server_Strategy_Factory '-ORBMaxRecvGIOPPayloadSize 268435456'"(Orb.cpp:741),9:Orb::init,1
0,51216,137,112,7,1331068607859,2880,2460,0:,615:advertising: name = Proxy.StorageServerManager as ior = IOR:010000004d00000049444c3a566572697461732f4e65744261636b75702f4469736b4d656469615365727665722f5354532f50726f78792f4150492f53746f726167655365727665724d616e616765723a312e3000000000010000000149434fab000000010102000b000000524c56494557504f52540000140600000b00000050726f7879563953564300631b00000014010f00525354bf7e564fe6df0c000000000001000000010000007003000000000000000800000001000000004f415401000000180000000100000001000100010000000100010509010100000000000300000027000000010000000e0000003135392e3139382e3136392e360014060b00000050726f7879563953564300(Orb.cpp:859),20:Orb::advertiseObject,1
0,51216,137,112,8,1331068607859,2880,2460,0:,606:advertising: name = Proxy.EventChannelManager as ior = IOR:010000004c00000049444c3a566572697461732f4e65744261636b75702f4469736b4d656469615365727665722f5354532f50726f78792f4150492f4576656e744368616e6e656c4d616e616765723a312e3000010000000149434fab000000010102000b000000524c56494557504f52540000140600000b00000050726f7879563953564300631b00000014010f00525354bf7e564fe6df0c000100000001000000020000007003000000000000000800000001000000004f415401000000180000000100000001000100010000000100010509010100000000000300000027000000010000000e0000003135392e3139382e3136392e360014060b00000050726f7879563953564300(Orb.cpp:859),20:Orb::advertiseObject,1
0,51216,137,112,9,1331068608062,2880,2460,0:,73:performing call with the only endpt available!(Endpoint_Selector.cpp:437),33:EndpointSelector::select_endpoint,1
0,51216,137,112,10,1331068609062,2880,2460,0:,73:performing call with the only endpt available!(Endpoint_Selector.cpp:437),33:EndpointSelector::select_endpoint,1
1,51216,137,112,11,1331068609062,2880,2460,0:,0:,28:OrbService::enableMonitoring,6,(100|A128:system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0'
OMG minor code (2), described as '*unknown description*', completed = NO
|)
0,51216,137,112,12,1331068788062,2880,2888,0:,73:performing call with the only endpt available!(Endpoint_Selector.cpp:437),33:EndpointSelector::select_endpoint,1
0,51216,137,112,13,1331068789062,2880,2888,0:,73:performing call with the only endpt available!(Endpoint_Selector.cpp:437),33:EndpointSelector::select_endpoint,1
1,51216,137,112,14,1331068789062,2880,2888,0:,0:,28:OrbService::enableMonitoring,6,(100|A128:system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0'
OMG minor code (2), described as '*unknown description*', completed = NO
|)
2,51216,137,112,15,1331068789062,2880,2888,0:,0:,0:,0,(74|)
2,51216,137,112,16,1331068789062,2880,2888,0:,0:,0:,0,(12|A10:ProxyV9SVC|)
2,51216,137,112,17,1331068789062,2880,2888,0:,0:,0:,0,(74|)

Thanks in Advance for any help, suggestions.

[Marianne]

This looks like a unified log in one of the other log folders, not bpcd. What exactly is the filename of this log file?

Can you confirm that bpcd and vnetd on the client are both LISTENING?

Which NetBackup and Symantec services are running on the client?

Can you telnet from master to vnetd port (13724) on client?

When doing bptestbpcd connection test, the source port is not what matters - only the destination port.

Please post output of :

bptestbpcd -client <client-name> -verbose -debug

[grosstech]

Hi Marianne:

    Thanks for the response.  The name of the log file is:   

51216-112-111789727-120306-0000000000.log. 

This is a peculiar situation where a machine was moved and currently cannot be administered remotely.  I am working on getting a administrator in the datacenter to work with the machine.  On the client

1. netstat -a will show bpcd and vnetd in a LISTENING state.  I don't believe that it will show the ports but ...

Should be able to telnet localhost 13724 and 13784 to validate.

2. Control Panel-> Administrative Tools -> Services for Netbackup and Symantec services.

I cannot telnet from the master to the vnetd port on the client.

C:\Windows\system32>telnet <client> 13724
Connecting To <client>...Could not open connection to the host, on port 13724:
Connect failed

If the client were not listening on the vnetd port this could happen.  Makes me suspicious of the install of the client software.

D:\Veritas\NetBackup\bin\admincmd>bptestbpcd -client <client_name> -verbose
<16>bptestbpcd main: Function ConnectToBPCD(rlviewport) failed: 25 cannot connect on socket

Thanks.

[Mark_Solutions]

OK - a couple of things here based on what you have said ...

1. you will need the vnetd port opend betweem the client and Master / Media Servers

2. Remove the client from the Master Server Host properties - Firewall section as this is for Media Servers behind a firewall

3. Add it to Master Server Host Properties - Client attributes. Once added select it and go to the connection options tan and set the top and bottom one (off the top of my head) to vnetd only

It should then work

[js88699]

I belive Mark has given you very good advive, but since I spent most of the afternoon looking at this, it's fresh in my mind.  Starting with 7.0.1 NetBackup wants to use pbx (1556) as the primary port.  If you can you may want to look at opening the PBX port on the firewall.  Yes it will fall back to vnetd, tested and confirmed that behaivior.  Here are two tech notes and a forum discussion I found today.

Ten second connection delays cause status 25 after upgrading mediaserver to NetBackup 7.1

http://www.symantec.com/business/support/index?page=content&id=TECH162303

NetBackup 7.0.1 and above attempts to use the PBX port for connections before falling back to vnetd, daemon, and legacy ports.

http://www.symantec.com/business/support/index?page=content&id=TECH136791

7.0.1 Clients making calls on 1556 (PBX)  (Forum discussion)

https://www-secure.symantec.com/connect/forums/701-clients-making-calls-1556-pbx

Just some extra info for you.  Good luck.

[Marianne]

The log that you are looking at is a unified log (all I could find for OID 112 is: Default OID; not in nblog.conf).

bpcd is a legacy log and is created with a filename <date>.log under <install-path>\veritas\netbackup\logs\bpcd\. The folder does not exist by default - it must be created.

If you cannot telnet to vnetd port, you need to speak to the firewall admins. Let them monitor connection attempts while you are doing bptestbpcd.
As pointed out above, PBX (1556) will first be tried, then vnetd (13724).
Minimum requirement is for vnetd to be open in both directions.
Please start adding 1556 to your firewall requests - seems NBU is moving towards PBX only...

[grosstech]

Sorry for the delay as this server is still not backing up.  I had actually done everything correctly.  The issue has been kicking around for months with a good deal of finger pointing.  Yesterday a tech rep from the Vendor that configured the server with the custom application made an appearance.  NBU will not work with the current configuration.  Will require reconfiguring which will void the current support contract which has cost $$$.  Thanks to everyone for their suggestions.  This is the first client which I have not been able to configure and the reason is not NBU.  Unfortunately, and I am open to suggestions I do not believe that there was a solution that was provided.  Many excellent suggestions which I had already pursued.  I'm open to suggestions if someone believes that my conclusions are incorrect.  I do believe in providing credit where due.  Thanks to everyone for attempting to help.   ---  mig

[Mark_Solutions]

Hi - just back off holiday and catching up on things ...

What is the reason that you say NBU will not work with the current configuration?

Please explain exactly what you have been told and what you actually have to work with here

Just trying to understand what the catch is to see if it possible to over come it

Thanks

[grosstech]

The intended NBU Windows client is a dedicated third party application software platform.  It "sits" in a DMZ which was/is intended to be heavily firewalled.  Networking folks claimed that they had temporarily disabled all firewall constraints in the interest of making the backups work.  Communication from both the master and media servers to vnetd and the pbx ports were never successful.  Eventually the application vendor was brought in and stated that they were aware that their product would not work with NBU without modifications to their server configuration.  What those modifications would have to be were never stated.  But, there was a warning that these type of modifications would void the support contract which is quite costly.  Thanks for attempting to assist.  -- mig

[Mark_Solutions]

OK - cant imagine what it could be - if NBU is installed on the client and vnetd port is open, plus hosts file entries on client for Master + Media and Master + Media for Client then adding that client to the Master Server Client Attributes section and in the Connection Options section selecting BPCD and Deamon Connection port to both be VNETD then that should be it!

No need for pbx - just vnetd to do the whole job

Oh well, we tried!