Forum Discussion

jbf23's avatar
jbf23
Level 2
9 years ago

Adding existing linux media server to DD storage server

I have the master server added to one of my Storage Servers (DataDomain). When I try adding media servers, I keep getting error:
Add credentials failed for media servers. Unknown error (-1)

 

After it fails this process stays hung on the media server:

root     39535 39534  0 Jun28 ?        00:00:04 /usr/openv/volmgr/bin/tpconfig -add -storage_server dom10.dss.comp.com -stype DataDomain -sts_user_id aoldomost -password 802d370276221a19ee98d510876e5bf19b6c09353be8c7ca2c343e609b468464117ccd9142de7ab80b1dd106175f4e4b54b615748113086f0b08be59a964790e -key 1207355938 -noverbose

Which logs should I be looking in for this? I saw one forum talking about the tpcommand log, which I created but no log appears.

**NOTE**
All servers were added to this storage server in the past, but we had a switch issue recently and were unable to connect to the DD. This issue was resolved and I can telnet to the DD again from the media servers, but still cannot add them to the Storage Server.

  • There are three common causes why a media server can't be added to the Credentials > Storage Server for Data Domain, unfortunately they all spit out the same error.  Those causes are: a) lack of network connectivity to the ddboost ports on the DD, or to the DD itself in general; b) bad ddboost username or password; and c) the media server is not in the ddboost authorized clients list on the DD.

    For a) You say you can telnet to the DD, but can you connect using the storage server name defined in NetBackup to the required ports 111, 2049 & 2052: https://community.emc.com/docs/DOC-33258

    For b) If the credentials are already known to the master server, you can use the GUI to simply add a checkmark next to the applicable media servers and it will re-use the credentials already in place.

    For c) Check "ddboost clients show config"; by default this could be open to all using an asterisk, but if it's not you have to add the hostname of your media server to the list.  What I normally do is add the asterisk to ddboost clients, connect my media server, find the media server name reported by "ddboost show connections", add that name to ddboost clients, remove the asterisk.

    As for the logs, start with nbstserv, nbemm & nbrmms in unified logging.  What I normally do, after making sure all OIDs reference the "Default" for DebugLevel and DiagnosticLevel:

    1. crank up unified logging with vxlogcfg -a -p 51216 -o Default -s DebugLevel=6
    2. rerun my task
    3. turn down unified logging with vxlogcfg -a -p 51216 -o Default -s DebugLevel=1
    4. Search the last minute of all unified logs for the media server with vxlogview -w "(prodid='NB' && prevtime='0:1:0')" -d all | less +/media-server-name

    Besides the physical switch being replaced, did anything change during the network fix?  IPs, names, firewalls, VLANs?  Was the switch config properly re-applied to the new switch?

    I do find it odd because even if network connectivity is lost, the NetBackup config should still show those media servers has having the storage server credentials.  I have some media servers who use ddboost over the WAN and they lose network access from time to time, but they never get removed from the storage server credentials.

    I like PatS729's suggestion, and taking that one step further you can stop the media servers completely, remove them from EMM using nbemmcmd -deletehost.  When you restart the media servers they will try to check in with EMM again, if that works then connectivity from master to media seems OK and you can try adding credentials as if they were brand new media servers.  This process may require you to remove the media servers from their storage units before you can delete them from EMM, but that's part of trying to do a clean add.

     

  • They were added, but after the switch issue was fixed, when I refreshed the storage server view, only the master was listed. Therefore I was trying to readd them

  • Have you check if connectivity between Master and Media is OK ?

    Do this test from master

    bptestbpcd -host media_server -verbose

    bptestnetconn -frap -s -v

    Do this from media

    bptestbpcd -host master_server

    nbemmcmd -getemmserver

    bptestnetconn -frap -s -v

  • There are three common causes why a media server can't be added to the Credentials > Storage Server for Data Domain, unfortunately they all spit out the same error.  Those causes are: a) lack of network connectivity to the ddboost ports on the DD, or to the DD itself in general; b) bad ddboost username or password; and c) the media server is not in the ddboost authorized clients list on the DD.

    For a) You say you can telnet to the DD, but can you connect using the storage server name defined in NetBackup to the required ports 111, 2049 & 2052: https://community.emc.com/docs/DOC-33258

    For b) If the credentials are already known to the master server, you can use the GUI to simply add a checkmark next to the applicable media servers and it will re-use the credentials already in place.

    For c) Check "ddboost clients show config"; by default this could be open to all using an asterisk, but if it's not you have to add the hostname of your media server to the list.  What I normally do is add the asterisk to ddboost clients, connect my media server, find the media server name reported by "ddboost show connections", add that name to ddboost clients, remove the asterisk.

    As for the logs, start with nbstserv, nbemm & nbrmms in unified logging.  What I normally do, after making sure all OIDs reference the "Default" for DebugLevel and DiagnosticLevel:

    1. crank up unified logging with vxlogcfg -a -p 51216 -o Default -s DebugLevel=6
    2. rerun my task
    3. turn down unified logging with vxlogcfg -a -p 51216 -o Default -s DebugLevel=1
    4. Search the last minute of all unified logs for the media server with vxlogview -w "(prodid='NB' && prevtime='0:1:0')" -d all | less +/media-server-name

    Besides the physical switch being replaced, did anything change during the network fix?  IPs, names, firewalls, VLANs?  Was the switch config properly re-applied to the new switch?

    I do find it odd because even if network connectivity is lost, the NetBackup config should still show those media servers has having the storage server credentials.  I have some media servers who use ddboost over the WAN and they lose network access from time to time, but they never get removed from the storage server credentials.

    I like PatS729's suggestion, and taking that one step further you can stop the media servers completely, remove them from EMM using nbemmcmd -deletehost.  When you restart the media servers they will try to check in with EMM again, if that works then connectivity from master to media seems OK and you can try adding credentials as if they were brand new media servers.  This process may require you to remove the media servers from their storage units before you can delete them from EMM, but that's part of trying to do a clean add.