β03-06-2014 05:16 AM
Hi All,
We have a newly setup media servers which is showing Active for Disk only instead of Active for Disk & Tape . SSO licence is active on the Media server
While configuration it shows the drive path and drives are visible from the command "vmoprcmd -d " for few seconds and then the drives go invisible as given below :
D:\Veritas\Volmgr\bin>vmoprcmd -d
We tried to force it from the command , but no use :
nbemmcmd -updatehost -machinename client_name -machinestateop set_tape_active -machinetype media -masterserver <Master server>
NBU Version: 7.5.0.6
OS : win2008 R2
I am attaching the LTID debug log.
Solved! Go to Solution.
β03-09-2014 02:52 PM
There are connectivity errors.
Media servers can go offline for tape (active for disk only) for many reasons including the following:
- Master failed to receive the hearbeats.
- Drive polling fails.
If ltid is exiting it's because it can't detect any devices on the system.
In this case ltid starts up, but detects a change in machine state:
17:36:15.121 [904.1212] <4> avrd: INITIATING
17:36:15.121 [6968.4420] <4> SendEmmHeartbeat: Detected change in MachineState...
17:36:15.121 [6968.4420] <4> SendEmmHeartbeat: ...clearing LOCAL_CONTROL bits
So apparently the heartbeats failed.
Every 5 minutes the media server will attempt to send the heart beat again... Every time after 17:36 - it Detects a change in machine state and tries to clear local bits and up the libraries.
At this point all updates fail:
18:06:15.442 [6968.4420] <16> emmlib_SetRobotStatus: (0) UpdateLibrary failed, emmError = 2007031, nbError = 0
18:06:15.442 [6968.4420] <16> ROBOT_CONNECT: (-) Translating EMM_ERROR_SQLNoDataFound(2007031) to 328 in the device management context
18:06:15.442 [6968.4420] <16> ROBOT_CONNECT: emmlib_SetRobotStatus() failed for robot 3 with error 328
18:06:15.442 [6968.4420] <4> SendEmmHeartbeat: Going Active failed - reset machine state.
I would suggest you check the following:
WINS or DNS server issues
A NIC going bad.
You might want to schedule a bptestnetconn every 10 minutes or so to see when the media sever is not available.
I woul suggest enabling the folloiwng logigng levels on the media server:
vxlogcfg -a -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -a -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
Make sure that the daemon directory is also available under volmgr\debug
On the maser - if space allows - run the following:
vxlogcfg -a -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -a -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -a -p 51216 -o 137-s DebugLevel=6 -s DiagnosticLevel=6
Check the resulting log files for issues connecting.
An example is in http://www.symantec.com/docs/TECH69625
*** Don't forget to remove logging when done.
Media server:
vxlogcfg -r -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -r -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
Master:
vxlogcfg -r -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -r -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -r -p 51216 -o 137-s DebugLevel=6 -s DiagnosticLevel=6
Good luck
After that happens this entity can no longer make any updates to the database and
β03-06-2014 05:27 AM
SSO licence is active on the Media server.
What about Library Based Tape Drives license?
Any chance a demo license key has expired and the above license is not installed?
Have you checked bpminlicense output on the media server?
So, you need 3 licenses on a media server:
Enterprise Server or Enterprise Client (depending on 'full' or SAN media server)
Library Based Tape Drives
Shared Storage Option
β03-07-2014 03:03 AM
Hi Marianne
Apology for delayed respone.
Below are the licences which are active on the new media server ( hostname02 ).
xxXx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xx xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xx
β03-07-2014 03:06 AM
Hello Marianne
Here is the details of license keys active on the new media server ..
β03-07-2014 03:19 AM
Hi Marianne,
I have attached a file for the active licenses on the media server.
β03-07-2014 03:53 AM
Okay, it's all there:
NetBackup Enterprise Server
Library Based Tape Drives
Shared Storage Option
The issue you are experiencing looks like a comms issue between EMM on the master and the media server(s) using PBX (port 1556).
Can you confirm that this port is open between master and media servers in both directions?
Windows firewall disabled on all Windows servers?
Is forward and reverse lookup working in all directions?
Please post output of the following commands on the master server:
nbemmcmd -listhosts -verbose
nbemmcmd -getemmserver
β03-07-2014 05:07 AM
Check you can connect to it from the Master
Open the admin console - go to Media and Device Management - Devices - Media Servers
Right click on the Media Server and select Stop / Re-Start Device Manager Service
If it connects then you should be Ok for comms, if not there is an issue
When you do this it is worth clicking Apply anyway as that usually kicks a new media server into life and sets it active for tape too
β03-07-2014 06:45 AM
Marianne's Answer :
Yes , Port 1556 is open bidirectionally Host file entries is been on both the servers.
Firewall is disabled .
All the media servers are listed in the output of "nbemmcmd -getemmserver" including the new one
MEDIA 7.5 Mediaserver02 Masterserver02 # server name has been edited
Output of "nbemmcmd -listhosts -verbose"
And yesterday's troubshooting
β03-09-2014 01:05 AM
License keys seems fine and comms also seems to be fine (at the time that you issued nbemmcmd commands).
Are these media server in a remote site? Or different vlan?
Anything that could cause intermittent comms issues?
EMM sends out regular 'heartbeat' comms to media servers. If media servers are slow or fail to respond in time, EMM will mark them as 'Active for disk'.
EMM heartbeat polling to media servers can be configured with "SCAN_HOST_STATUS_INTERVAL" .
See http://www.symantec.com/docs/TECH70827
The real solution will be to troubleshoot the intermittent network issues between the master and media server.
A complete restart of NBU (incl PBX) on master and media servers may also be a good idea.
β03-09-2014 07:52 AM
β03-09-2014 02:52 PM
There are connectivity errors.
Media servers can go offline for tape (active for disk only) for many reasons including the following:
- Master failed to receive the hearbeats.
- Drive polling fails.
If ltid is exiting it's because it can't detect any devices on the system.
In this case ltid starts up, but detects a change in machine state:
17:36:15.121 [904.1212] <4> avrd: INITIATING
17:36:15.121 [6968.4420] <4> SendEmmHeartbeat: Detected change in MachineState...
17:36:15.121 [6968.4420] <4> SendEmmHeartbeat: ...clearing LOCAL_CONTROL bits
So apparently the heartbeats failed.
Every 5 minutes the media server will attempt to send the heart beat again... Every time after 17:36 - it Detects a change in machine state and tries to clear local bits and up the libraries.
At this point all updates fail:
18:06:15.442 [6968.4420] <16> emmlib_SetRobotStatus: (0) UpdateLibrary failed, emmError = 2007031, nbError = 0
18:06:15.442 [6968.4420] <16> ROBOT_CONNECT: (-) Translating EMM_ERROR_SQLNoDataFound(2007031) to 328 in the device management context
18:06:15.442 [6968.4420] <16> ROBOT_CONNECT: emmlib_SetRobotStatus() failed for robot 3 with error 328
18:06:15.442 [6968.4420] <4> SendEmmHeartbeat: Going Active failed - reset machine state.
I would suggest you check the following:
WINS or DNS server issues
A NIC going bad.
You might want to schedule a bptestnetconn every 10 minutes or so to see when the media sever is not available.
I woul suggest enabling the folloiwng logigng levels on the media server:
vxlogcfg -a -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -a -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
Make sure that the daemon directory is also available under volmgr\debug
On the maser - if space allows - run the following:
vxlogcfg -a -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -a -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -a -p 51216 -o 137-s DebugLevel=6 -s DiagnosticLevel=6
Check the resulting log files for issues connecting.
An example is in http://www.symantec.com/docs/TECH69625
*** Don't forget to remove logging when done.
Media server:
vxlogcfg -r -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -r -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
Master:
vxlogcfg -r -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -r -p 51216 -o 156 -s DebugLevel=6 -s DiagnosticLevel=6
vxlogcfg -r -p 51216 -o 137-s DebugLevel=6 -s DiagnosticLevel=6
Good luck
After that happens this entity can no longer make any updates to the database and
β04-03-2014 03:03 AM
Hi All ,
Thanks for your valuable contributions.
This issue has been resolved after OS rebuild and did persistent binding of the drive.
Problem : Drives were attached to the Library which has some issues.