globalDB hostname has not been set (MM status 4)

[MattS]

I'm getting the error below when trying to add the tape drives of 6 new san media servers:

globalDB hostname has not been set (MM status 4)

This error pops up after your select the media servers in the Device Configuration Wizard.

I have added the media servers using the following command and then restarted the services:

./nbemmcmd -addhost -machinename cntpadb2a -machinetype media -masterserver tbstpbk2 -operatingsystem linux -netbackupversion 6

 I cant find anything on this.  Thanks!

Matt

edit: SORRY!  NB6.0MP6, Solaris 9 master and the media servers are Oracle RedHat Ent Linux

Message Edited by MattS on 10-16-2008 06:57 AM

[Manoj_Siricilla]

Matts,

We need to do some troubleshooting here.

run this command /usr/openv/netbackup/bin/admincmd/nbemmcmd -listhosts

Do you see your server listed here as

media  cntpadb2a ?

If No, then your server is not registered as a Media server and do not manually try to register, we'll have netbackup automatically do this.

On the Media server, make sure the bp.conf has the SERVER, CLIENT_NAME, and EMMSERVER entries and all correct Ip address have been populated in /etc/hosts or you might have them in DNS.

Now, If everything is ok, restarting the services on the Media server should automatically register the server as Media server in the EMM.

If this part is done we can move to the tape config.

You can run the following command on the Media server to automatically configure tape devices

/usr/openv/volmgr/bin/tpautoconf -a

If this command fails, let me know, we can troubleshoot, If it completes then we need to start the Media Manger device daemon

/usr/openv/volmgr/bin/ltid

after which you can run the following command to see if tape devices are discovered and configured.

/usr/openv/volmgr/bin/vmoprcmd -d ds

Let me know the outcome.

[MattS]

/usr/openv/volmgr/bin/tpautoconf -a

gave the following error:

 Failed opening the device mappings file

Thanks!

[Manoj_Siricilla]

Did you add the media server entry to the bp.conf on the master

SERVER = <media server hostname> ?

Also, from the media server can you see you can telnet on port 1556

#telnet <master server hostname> 1556

from the master server

#telnet <media server hostname > 1556

[MattS]

From the media server i CANNOT telnet on that port.  But from the master to media i can.

I can telenet on bpcd port both ways.

I have never installed a media server threw a firewall so i imagine this is the issue?

Let me know, already got the firewall admins working on that port.

Thanks again!

[Manoj_Siricilla]

Starting 6.x all we need for a media server working is the following ports

vnetd - 13724

bpcd  - 13782

pbx -1556

we need bi-directional communication between the master and media servers.

[MattS]

Ok all those ports are now available both directions and i still get the error :(

Anything else you can think of?

[MattS]

/usr/openv/volmgr/bin/tpautoconf -a

Still returns:

 Failed opening the device mappings file

Whats odd is i run that same command on the other nodes in this setup (which all should be the same) i get:

  /usr/openv/volmgr/bin/tpautoconf: symbol lookup error: /usr/openv/lib/libVnbatST.so: undefined symbol: ovgetlocale

[MattS]

PBX doesnt need to be installed for a SAN media server does it?

edit: nevermind it is installed

Message Edited by MattS on 10-16-2008 12:39 PM

[Manoj_Siricilla]

Matts,

ICS needs to be installed on all servers running as Media servers as they communicate with the EMM.

can we stop and start the services on media server including the pbx, as the pbx picks up the config when it starts and we need to reload the addresses that are used for communication.

/usr/openv/netbackup/bin/goodies/netbackup stop

/opt/VRTSpbx/bin/vxpbx_exchanged stop

 now lets restart

 pbx has to be started before you start netbackup.

/opt/VRTSpbx/bin/vxpbx_exchanged start

/usr/openv/netbackup/bin/goodies/netbackup start

[MattS]

I have been doing those steps for a bit now... seems i have 2 problems.

First node which has a fresh install of the netbackup software gets this:

[root@cntpadb2a ~]# /usr/openv/volmgr/bin/tpautoconf -a
Failed opening the device mappings file

Other 5 nodes get this:

  [root@cntpadb2b admincmd]# /usr/openv/volmgr/bin/tpautoconf -a
/usr/openv/volmgr/bin/tpautoconf: symbol lookup error: /usr/openv/lib/libVnbatST.so: undefined symbol: ovgetlocale

I think the issue between the 2 is a license issue.  I have a working Oracle rac with each node as a media server and we are using a different license out of our "gold" keys.  So i put that key on the first node and it seems to work better.  For instance when i start the other 5 nodes i get the following:

[root@cntpadb2b admincmd]# /usr/openv/netbackup/bin/goodies/netbackup startNetBackup Database Server started./usr/openv/netbackup/bin/nbnos: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Notification Service started./usr/openv/netbackup/bin/nbemm: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Enterprise Media Manager started./usr/openv/netbackup/bin/nbrb: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Resource Broker started.Rebuilding device nodes.Media Manager daemons started.NetBackup request daemon started./usr/openv/netbackup/bin/bpcompatd: symbol lookup error: /usr/openv/lib/libVnbatST.so: undefined symbol: ovgetlocaleNetBackup compatibility daemon started./usr/openv/netbackup/bin/nbjm: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Job Manager started./usr/openv/netbackup/bin/nbpem: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Policy Execution Manager started./usr/openv/netbackup/bin/nbsl: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Service Layer started./usr/openv/netbackup/bin/nbsvcmon: symbol lookup error: /usr/openv/lib/libVnbat.so: undefined symbol: ovgetlocaleNetBackup Service Monitor started.[root@cntpadb2b admincmd]# /usr/openv/volmgr/bin/tpautoconf -a/usr/openv/volmgr/bin/tpautoconf: symbol lookup error: /usr/openv/lib/libVnbatST.so: undefined symbol: ovgetlocale

I also get the following on the first node from the following commands:

[root@cntpadb2a ~]# /usr/openv/volmgr/bin/tpext -get_dev_mappings_ver
Segmentation fault


Whereas on my other RAC cluster which is functioning i get this:

[root@tbstpdb23a admincmd]# /usr/openv/volmgr/bin/tpext -get_dev_mappings_ver
device mappings version in the EMM database is 1.49.12.10
device mappings version from the local file is 1.49.12.10
Local device mappings file is up-to-date

[Manoj_Siricilla]

Matts,

Lets focus only on cntpadb2a

- Can you stop and restart the pbx and netbackup services ?

- can you telnet on port 1556 from the cntpadb2a server to your master server?

Now we will enable verbose logging

touch /usr/openv/volmgr/vm.conf ( if the file doesn't exist)

echo "VERBOSE = 5" >> /usr/openv/volmgr/vm.conf

mkdir /usr/openv/volmgr/debug/tpcommand ( if the directory doesn't exist)

Can you now run the command /usr/openv/volmgr/bin/tpautoconf -a ?

And paste the lines from the log file generated from /usr/openv/volmgr/debug/tpcommand/log.101608

[MattS]

Yes to your first 2 questions.  I even had my firewall admin make sure there was nothing being dropped between the master and media servers.

Here is the log file:

[root@cntpadb2a tpcommand]# cat log.101608
19:54:04.889 [8998] <4> tpautoconf_main: /usr/openv/volmgr/bin/tpautoconf -a
19:54:04.890 [8998] <2> mm_getnodename: cached_hostname cntpadb2a.corp.local, cached_method 3
19:54:04.913 [8998] <2> mm_getnodename:  (3) hostname cntpadb2a (from mm_master_config.mm_server_name)
19:54:04.913 [8998] <4> InitThisHostName: ThisHost is cntpadb2a
19:54:04.913 [8998] <16> MmDeviceMappings::InitializeMappings
        : <MmDeviceMappings.cpp:222> failed to open mappings file. errno = 2
19:54:04.913 [8998] <16> tpautoconf_main: Unable to open the device mappings table, status = 1

 

 I found this link before i eft work, what are your thoughts on this?  Thanks again for all your help!

http://seer.entsupport.symantec.com/docs/308718.htm

Message Edited by MattS on 10-16-2008 04:58 PM

[Manoj_Siricilla]

Matts,

Symantec once in two months adds supports to new tape devices. To take advantage of those newly supported tape devices, we need to update the EMM database with the external_types.txt they provide.

Looking at the log, I see you do not have the device_mappings.txt file in the location /usr/openv/var/global.

If don't find a file, can you get it netbackup installation disk or the master server anc copy it to the location /usr/openv/var/global?

We need not load anything into the EMM, can we run now /usr/openv/volmgr/bin/tpautoconf -a?

[MattS]

Now i get a segmentation fault when that runs.  Have stop and started NB and PBX as well.

The log file output is better though:

 21:09:59.572 [14451] <4> tpautoconf_main: /usr/openv/volmgr/bin/tpautoconf -a
21:09:59.572 [14451] <2> mm_getnodename: cached_hostname cntpadb2a.corp.local, cached_method 3
21:09:59.596 [14451] <2> mm_getnodename:  (3) hostname cntpadb2a (from mm_master_config.mm_server_name)
21:09:59.596 [14451] <4> InitThisHostName: ThisHost is cntpadb2a
21:09:59.603 [14451] <4> tpautoconf_main: successfully intialized the device mappings table

[Manoj_Siricilla]

Matts,

we cannot do with such little logs, we need to enable verbose logging,

can you run the following commands?

/usr/openv/netbackup/bin/vxlogcfg -p 51216 -o 137 -a -s DebugLevel=3;DiagnosticLevel=6

and now rename the file /usr/openv/volmgr/debug/tpcommand/log.101608 /usr/openv/volmgr/debug/tpcommand/log.101608.old

run the command

/usr/openv/volmgr/bin/tpautoconf -a

and paste the log, we'll take it from there.

Message Edited by Manoj Siricilla on 10-16-2008 10:03 PM

[MattS]

I just realized something.  This is a 64bit server and the linux server patch i applied was not the 64bit version.  Could this be causing my problems? 

[Manoj_Siricilla]

It will certainly cause problems, that is the reason it is causing segmentation faults.

Can you get that done?

and also if the problem doesn't resolve, we need logging enabled for all these three , ignore my previous message for logging.

vxlogcfg -p 51216 -o 137 -a -s DebugLevel=6;DiagnosticLevel=6

vxlogcfg -p 51216 -o 156 -a -s DebugLevel=6;DiagnosticLevel=6

vxlogcfg -p 51216 -o 143 -a -s DebugLevel=6;DiagnosticLevel=6

Message Edited by Manoj Siricilla on 10-16-2008 10:13 PM

[MattS]

Cant believe i missed this.  Uninstalling the other mp and installing the new one now.

[MattS]

Uninstalled all MPs and resintalled with the 64bit server version.

Still getting the same error, segmentation fault.  The logs look the same too :(

23:16:07.849 [18760] <4> tpautoconf_main: /usr/openv/volmgr/bin/tpautoconf -a
23:16:07.849 [18760] <2> mm_getnodename: cached_hostname cntpadb2a.corp.local, cached_method 3
23:16:07.873 [18760] <2> mm_getnodename:  (3) hostname cntpadb2a (from mm_master_config.mm_server_name)
23:16:07.873 [18760] <4> InitThisHostName: ThisHost is cntpadb2a
23:16:07.880 [18760] <4> tpautoconf_main: successfully intialized the device mappings table