NBU - SUN ACS and RPC problems

[Kenneth_Hansen]

Hi folks, 

I've been looking into this problem before and it's not been easy to figure out.
What I'm looking for is to control which ports that NBU and ACS uses to communicate.
This relates to both windows and solaris in our environment.

This has been done:

NBU media servers
vm.conf:
ACS_TCP_RPCSERVICE
ACS_SSI_INET_PORT = HOST 30032
ACS_CSI_HOSTPORT = HOST 30031

ACSLS Server

Enable CSI to be used behind a firewall (user-defined inbound port) [TRUE]:
Port number used by the CSI to receive incoming ACSLS requests [30031]:
CSI support for RPC using the TCP protocol is enabled [TRUE]: ( Default , whathappens if i turn this of? )
CSI support for RPC using the UDP protocol is enabled [TRUE]: ( Default, whathappens if i turn this of? )

Problem to solve:
RPC demands that a port range is openv in FW to, default 1025 - 65554 or something like that.
That I would like to see is RPC disabeled, reason for this is to be able to control down to a single port ( or at least only a few ports ) the communication between ACSLS and NBU servers.

Question:
Have anyone disabled RPC on ACSLS and NBU? How did you do it? What is your experience?

Please help !!!

[marekkedzierski]

Try with this:

MM_SERVER_NAME = MEDIA_SERVER_NAME
CSI_HOSTNAME = ACSLS_SERVER_NAME
SSI_HOSTNAME = MEDIA_SERVER_NAME
ACS_TCP_RPCSERVICE
ACS_SSI_INET_PORT = ACSLS_SERVER_NAME 32001
ACS_CSI_HOSTPORT = ACSLS_SERVER_NAME 32001
CSI_RETRY_TIMEOUT = 10
CSI_RETRY_TRIES = 20

[Kenneth_Hansen]

Thanks for that information.

But as far as I can see, this does not do any thing with RPC, I've added the extra configuration lines. And I've tested on a media server, RPC still openes a different port.
Updated vm.conf

MM_SERVER_NAME = HOST
SSO_SCAN_ABILITY = 0
CSI_HOSTNAME = ACS HOST
SSI_HOSTNAME = NBU HOST
ACS_TCP_RPCSERVICE
ACS_SSI_INET_PORT = ACS HOST 30031
ACS_CSI_HOSTPORT = ACS HOST 30031
CSI_RETRY_TIMEOUT = 10
CSI_RETRY_TRIES = 20

RPC port used after a reboot or media server:
1073741824 2 tcp 32911

Please advice :)

[Marianne]

See this TechNote for NBU and ACSLS configuration:

http://seer.entsupport.symantec.com/docs/306739.htm

Extract :

The following requires 3rd party configuration. Refer to SUN/StorageTek documentation and/or support channel for assistance.
Corresponding settings on the ACSLS server must match the settings in the vm.conf file. For example, in a typical ACSLS firewall configuration, the following settings would be changed:

* Changes to alter the use of TCP protocol - Set to TRUE - Firewall-secure ACSLS runs across  TCP.

* Changes to alter the use of UDP protocol - Set to FALSE - Firewall-secure ACSLS runs across TCP.

* Changes to alter the use of the portmapper - Set to NEVER -Ensures that the ACSLS server will not make any queries of the portmapper on the client platform.

* Enable CSI to be used behind a firewall- Set to TRUE - Allows specification of a single port to be used by the ACSLS server.

* Port number used by the CSI.

[Kenneth_Hansen]

This is perfect, exactly what I'm looking for.

I do have one more question though, windows servers connecting to ACS based robotic uses SUN Libattach.
Do you or any one have a sample configuration to send me?
Screen shot would be great.

Regarding libattach, when disabling port mapper on acsls, does sun rpc port mapper need to e installed with libattach?

Thank you :)

[Nicolai · Accepted solution]

Netbackup and the ACSLS firewall  feature, step by step description:

http://www.mass.dk/netbackup/guides/48-netbackup-and-the-acsls-firewall-feature.html

Please note: The firewall must be configured to allow session being initiated from both sides. ACSLS does not answer a request in the same session, but reply by initiating a new session.

[Kenneth_Hansen]

I've tested configuration changes today. But it did not "work" to my expectation. 

Netbackup master
Run command netbackup stop, verified all have stoped. 
rpcinfo -d 1073741824 1
rpcinfo -d 1073741824 2

vm.conf
ACS_SSI_HOSTNAME = MASTER SERVER
ACS_TCP_RPCSERVICE
ACS_SSI_INET_PORT = ACSLSHOST 30031
ACS_CSI_HOSTPORT = ACSLSHOST 30031

Guess this means that media server connects to acsls on port 30031 only..

On my ACSLS server:
kill.acsss

* Changes to alter the use of TCP protocol - Set to TRUE - Firewall-secure ACSLS runs across TCP. OK
* Changes to alter the use of UDP protocol - Set to FALSE - Firewall-secure ACSLS runs across TCP. OK
* Changes to alter the use of the portmapper - Set to NEVER -Ensures that the ACSLS server will not make any queries of the portmapper on the client platform. OK
* Enable CSI to be used behind a firewall- Set to TRUE - Allows specification of a single port to be used by the ACSLS server. OK
* Port number used by the CSI. OK 30031

rc.acsss

rpcinfo -p on acsls server :
1073741824 1 tcp 32776
300031 2 tcp 30031

So acsls listens at port 30031.

Netbackup start
rpcinfo -p
1073741824 2 tcp 35083 ????  ( shouldn't this be 30031 )

I've looked at the two link's provided in this thread 
http://www.mass.dk/netbackup/guides/48-netbackup-and-the-acsls-firewall-feature.html
http://seer.entsupport.symantec.com/docs/306739.htm

What Nicolai sais, does that mean that acsls would use RPC to connect back to netbackup media server on other ports than 30031?

My goal is to limit all traffic between nbu media servers and acsls both ways..
Meaning I need to limit the port that acsls connect back to nbu media servers.

Any one?

[Kenneth_Hansen]

Maximum age in seconds of pending requests in CSI
request queue [172800]:

Number of seconds between successive retries [4]:

Number of retries for the CSI before a timeout condition occurs [5]:

Changes to alter use of the TCP protocol will not take effect until the product is restarted. CSI support for RPC using the TCP protocol is enabled [TRUE]:

Changes to alter use of the UDP protocol will not take effect until the product is restarted. CSI support for RPC using the UDP protocol is enabled [FALSE]:

Changes to alter use of the port mapper will not take effect until the product is restarted. Enable port mapper: (ALWAYS / NEVER / IF_DUAL_LAN_NOT_ENABLED) [NEVER]:

Number of ACSSURR persistent processes that should be started [0]:

TCP/IP port number that the ACSLS surrogate (ACSSURR) socket will listen on for requests from a gateway system [50300]:

Number of seconds to wait for data packets to be read on surrogate/gateway socket [10]:

Number of minutes to wait before deleting a stale queue entry [5]:

Automatically start CSCI at ACSLS startup (TRUE/FALSE) [FALSE]:

Enable CSI to be used behind a firewall (user-defined inbound port) [TRUE]:

Port number used by the CSI to receive incoming ACSLS requests [30031]:

[Kenneth_Hansen]

Please need some help here..

[Marianne]

Are you getting errors when trying to connect? Are you just getting a 1-line output from rpcinfo?

I have just checked a media server that's working 100%: (30031 only port open in both directions between media server and acsls)

# cat vm.conf  (media server has one NIC only, therefore no ACS_SSI_HOSTNAME)
MM_SERVER_NAME = media01
ACS_TCP_RPCSERVICE
ACS_CSI_HOSTPORT = acsls 30031
ACS_SSI_INET_PORT = acsls 30031


bpps -x output:
MM Processes
------------
    root 18440 18012   0   Jul 29 ?           6:56 avrd
    root 18478 18219   0   Jul 29 ?           0:04 acssel -s 13740
    root 18316 18012   0   Jul 29 ?           0:35 tldd
    root 18020     1   0   Jul 29 ?           2:15 vmd
    root 18012     1   0   Jul 29 ?           2:47 /usr/openv/volmgr/bin/ltid
    root 18518     1   0   Jul 29 ?           1:41 tldcd
    root 18219 18012   0   Jul 29 ?           0:02 acsd
    root 18810 18219   0   Jul 29 ?           1:26 acsssi 13741

 # rpcinfo -p
   program vers proto   port  service
    100024    1   udp  32772  status
    100024    1   tcp  32771  status
    100133    1   udp  32772
    100133    1   tcp  32771
1073741824    1   tcp  32772
    100021    1   udp   4045  nlockmgr
    100021    2   udp   4045  nlockmgr
    100021    3   udp   4045  nlockmgr
    100021    4   udp   4045  nlockmgr
    100021    1   tcp   4045  nlockmgr
    100021    2   tcp   4045  nlockmgr
    100021    3   tcp   4045  nlockmgr
    100021    4   tcp   4045  nlockmgr
    100001    2   udp  32773  rstatd
    100001    3   udp  32773  rstatd
    100001    4   udp  32773  rstatd
    100068    2   udp  32774
    100068    3   udp  32774
    100068    4   udp  32774
    100068    5   udp  32774
    100083    1   tcp  32775
    100002    2   tcp  32776  rusersd
    100002    3   tcp  32776  rusersd
    100002    2   udp  32775  rusersd
    100002    3   udp  32775  rusersd
    100011    1   udp  32776  rquotad
    300598    1   udp  32779
    300598    1   tcp  32777
 805306368    1   udp  32779
 805306368    1   tcp  32777
    100249    1   udp  32780
    100249    1   tcp  32778
1073741824    2   tcp  30031

[Kenneth_Hansen]

And thank you for your answer.

No there is no problem connecting as of now, but firewall has a very scary opening between NBU master and mediaserver - ACSLS.
Every works, but I need to be able to close down firewall on our internal network.

I'm testing on a newly installed solaris 10 media server.
vm.conf 

ACS_TCP_RPCSERVICE
ACS_CSI_HOSTPORT = acsls 30031
ACS_SSI_INET_PORT = acsls 30031

/usr/openv/netbackup/bin/goodies/ netbackup stop and start

NB Processes
------------
root 20983 1 0 13:10:41 ? 0:00 /usr/openv/netbackup/bin/bpcompatd
root 20994 1 0 13:10:44 ? 0:01 /usr/openv/netbackup/bin/nbrmms
root 21006 1 0 13:10:45 ? 0:00 /usr/openv/netbackup/bin/nbsl
root 21013 1 0 13:10:45 ? 0:00 /usr/openv/netbackup/bin/nbsvcmon


MM Processes
------------
root 21016 21014 0 13:10:49 ? 0:00 acssel -s 13740
root 20967 1 0 13:10:40 ? 0:00 /usr/openv/volmgr/bin/ltid
root 20975 1 0 13:10:40 ? 0:00 vmd
root 21014 20967 0 13:10:47 ? 0:00 acsd
root 21015 20967 0 13:10:49 ? 0:00 avrd
root 21024 21014 0 13:10:54 ? 0:00 acsssi 13741


Shared Symantec Processes
-------------------------
root 611 1 0 juni 01 ? 0:00 /opt/VRTSpbx/bin/pbx_exchange



rpcinfo -p

program vers proto port service
100000 4 tcp 111 rpcbind
100000 3 tcp 111 rpcbind
100000 2 tcp 111 rpcbind
100000 4 udp 111 rpcbind
100000 3 udp 111 rpcbind
100000 2 udp 111 rpcbind
100024 1 udp 32791 status
100024 1 tcp 32779 status
100133 1 udp 32791
100133 1 tcp 32779
100021 1 udp 4045 nlockmgr
100021 2 udp 4045 nlockmgr
100021 3 udp 4045 nlockmgr
100021 4 udp 4045 nlockmgr
100021 1 tcp 4045 nlockmgr
100021 2 tcp 4045 nlockmgr
100021 3 tcp 4045 nlockmgr
100021 4 tcp 4045 nlockmgr
100229 1 tcp 32784 metad
100229 2 tcp 32784 metad
100001 2 udp 32792 rstatd
100001 3 udp 32792 rstatd
100001 4 udp 32792 rstatd
100068 2 udp 32794
100068 3 udp 32794
100068 4 udp 32794
100068 5 udp 32794
100083 1 tcp 32786
100422 1 tcp 32788
100242 1 tcp 32790 metamedd
100230 1 tcp 32792 metamhd
100002 2 tcp 32794 rusersd
100002 3 tcp 32794 rusersd
100002 2 udp 32796 rusersd
100002 3 udp 32796 rusersd
100011 1 udp 32798 rquotad
300598 1 udp 32804
300598 1 tcp 32800
805306368 1 udp 32804
805306368 1 tcp 32800
100249 1 udp 32806
100249 1 tcp 32802
1073741824 2 tcp 51117  <------------- Still not correct.
Note!! During this test acsls is not stoped and started again.

ifconfig -a

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
 inet xxx.x.x.xxx netmask ffffff00 broadcast 146.2.4.255
ether 0:14:4f:eb:bd:80
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
inet6 ::1/128
e1000g0: flags=2004841<UP,RUNNING,MULTICAST,DHCP,IPv6> mtu 1500 index 2
inet6 fe80::214:4fff:feeb:bd80/10
ether 0:14:4f:eb:bd:80

Ip adress is masked.

Could it be something on acs side or related to this server having more than 1 nic interface?

[Marianne]

I see only one interface - e1000g0, not true? If output of 'hostname' is the same as the hostname assigned to e1000g0, then there should be no problem. If servers have multiple interfaces and a non-default interface is used to connect to ACSLS, we add 'ACS_SSI_HOSTNAME = <interface-name>' to vm.conf.

Also add VERBOSE entry to vm.conf to assist with troubleshooting (comms with ACSLS will be logged in /var/adm/messages).

As per Nicolai's web site - after vm.conf entries are updated, you have to ensure that all NBU daemons processes are stopped:
3: Stop Netbackup on the media server - Make sure to kill all daemons not stopped. Especially acsd and acssi NEEDS to be killed for Netbackup 5.x. (6.x as well...  use bp.kill_all to ensure all daemons/processes are stopped)

4: Delete any previous registered RPC services on the Netbackup servers. A stop/start of Netbackup may not do the job.

#rpcinfo -d 1073741824 1
#rpcinfo -d 1073741824 2



Test comms by running robtest in one window and monitor port comms using 'snoop' in another window.

[Nicolai]

Hi Kenneth & Marianne

One thing that usually go wrong is the firewall opening. The firewall MUST be configured to allow traffic to be INITIATED from both sides. statefu firewall inspecting is not enough.

On the ACSLS side verify port operating with "snoop {IP of Netbackup server}. If the firewall isn't configured properly you see ACSLS first trying port 30031 and then fall back on the old style RPC port 111

[Nicolai]

Kill every Netbackup daemon on the Netbackup server.

If I do a rpcinfo I get:

  program vers proto   port
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  32778  status
    100021    1   udp  32778  nlockmgr
    100021    3   udp  32778  nlockmgr
    100021    4   udp  32778  nlockmgr
    100024    1   tcp  49146  status
    100021    1   tcp  49146  nlockmgr
    100021    3   tcp  49146  nlockmgr
    100021    4   tcp  49146  nlockmgr
      4397    1   udp    654
      4397    1   tcp    655
1073741824    2   tcp  30031  <--- This is what you wan't to see.

[Nicolai]

During some poor man ACSLS HA testing I got same issue as Kenneth.  rpcinfo -p never showed port 30031 but all sort of different ports .  No Storage unit was created for that media server, as it was testing purely.  But when I created a  storage unit  - BINGO - Issue disappeared.

Really strange .....