12-03-2014 06:20 AM
Some of the backups in our environment are failing with 2027 error code.
Netbackup media server is available
Currently backups are runing using the drives in this media server
Please find the log for the backup failure
12/03/2014 09:05:06 - Info nbjm (pid=23658736) starting backup job (jobid=3044394) for client sapbwsv-nb, policy Ora_RMAN_BWS, schedule Arch_BWS
12/03/2014 09:05:06 - Info nbjm (pid=23658736) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=3044394, request id:{62938BE6-7AF5-11E4-A27D-665E5F7F0000})
12/03/2014 09:05:06 - requesting resource media-server-nb-hcart3-robot-tld-3
12/03/2014 09:05:06 - requesting resource sun107-nb.NBU_CLIENT.MAXJOBS.sapbwsv-nb
12/03/2014 09:05:06 - requesting resource sun107-nb.NBU_POLICY.MAXJOBS.Ora_RMAN_BWS
12/03/2014 09:05:06 - Waiting for scan drive stop STK.T10000A.013, Media server: media-server-nb
12/03/2014 09:07:06 - Info nbjm (pid=23658736) Waiting in NetBackup scheduler work queue on server sun107-nb
12/03/2014 09:07:13 - Error nbjm (pid=23658736) NBU status: 2027, EMM status: Media server is not active
Media server is not active (2027)
12-03-2014 06:26 AM
Let's see output of:
vmoprcmd
Is the media server down?
12-03-2014 06:55 AM
vmoprcmd output
DRIVE STATUS
Drv Type Control User Label RecMID ExtMID Ready Wr.Enbl. ReqId
0 hcart3 TLD - No - 0
0 hcart3 TLD - No - 0
1 hcart3 TLD - No - 0
1 hcart3 TLD - No - 0
2 hcart3 TLD - No - 0
2 hcart3 TLD - No - 0
3 hcart3 TLD Yes I00876 I00876 Yes Yes 0
3 hcart3 TLD Yes I00876 I00876 Yes Yes 0
4 hcart3 TLD - No - 0
4 hcart3 TLD - No - 0
5 hcart3 TLD Yes I00745 I00745 Yes Yes 0
5 hcart3 TLD Yes I00745 I00745 Yes Yes 0
6 hcart3 TLD Yes I00765 I00765 Yes Yes 0
6 hcart3 TLD Yes I00765 I00765 Yes Yes 0
7 hcart3 TLD - No - 0
7 hcart3 TLD - No - 0
8 hcart3 TLD - No - 0
8 hcart3 TLD - No - 0
9 hcart3 TLD - No - 0
9 hcart3 TLD - No - 0
10 hcart2 TLD - No - 0
10 hcart2 TLD - No - 0
11 hcart3 TLD Yes I00883 I00883 Yes Yes 0
11 hcart3 TLD Yes I00883 I00883 Yes Yes 0
ADDITIONAL DRIVE STATUS
Drv DriveName Shared Assigned Comment
0 STK.T10000A.011 Yes -
0 STK.T10000A.011 Yes -
1 STK.T10000A.013 Yes -
1 STK.T10000A.013 Yes -
2 STK.T10000A.014 Yes -
2 STK.T10000A.014 Yes -
3 STK.T10000A.006 Yes media-server
3 STK.T10000A.006 Yes media-server
4 STK.T10000A.010 Yes -
4 STK.T10000A.010 Yes -
5 GRV_T10000_02 Yes -
5 GRV_T10000_02 Yes -
6 GRV_T10000_03 Yes media-server
6 GRV_T10000_03 Yes media-server
7 STK.T10000A.004 Yes -
7 STK.T10000A.004 Yes -
8 STK.T10000A.015 Yes -
8 STK.T10000A.015 Yes -
9 STK.T10000A.073 Yes -
9 STK.T10000A.073 Yes -
10 GRV_LTO3_05 Yes -
10 GRV_LTO3_05 Yes -
11 STK.T10000A.009 Yes media-server
11 STK.T10000A.009 Yes media-server
12-03-2014 06:56 AM
attached output of vmoprcmd
12-04-2014 10:27 AM
You need to run the vmoprcmd command without any switches... This will provide the details of the shared drives, server connections and status.
12-16-2014 08:49 AM
Sorry for my late reply.
Please find the output without any switches
12-16-2014 08:30 PM
Use "nbemmcmd -listhosts -verbose" to check the status of your media server. An active media server looks like this:
mediaserver1
ClusterName = ""
MachineName = "mediaserver1"
FQName = "mediaserver1"
LocalDriveSeed = ""
MachineDescription = ""
MachineFlags = 0x77
MachineNbuType = media (1)
MachineState = active for tape and disk jobs (14)
MasterServerName = "masterserver"
NetBackupVersion = 7.6.0.3 (760300)
OperatingSystem = linux (16)
ScanAbility = 5
If in the MachineState it's not active for tape, it's a problem. Use this to change the state:
nbemmcmd -updatehost -machinestateop set_tape_active -machinename mediaserver1 -machinetype media -masterserver masterserver
If the updatehost command does not work either, you need to make sure your master server can connect (ping, nslookup, bptestbpcd) to the media server.
01-14-2015 04:01 AM
Multiple interfaces on the media server?
vmoprcmd output shows drives in use on media-server.
Job details shows a problem with media-server-nb.
Curious to see output of 'nbemmcmd -listhosts -verbose'.
This will show us how 'media-server' is configured in EMM.
02-17-2015 10:49 PM
Jobs are still failing with 2027:
02/17/2015 04:10:38 - Error nbjm (pid=23658736) NBU status: 2027, EMM status: Media server is not active
Media server is not active (2027)
Checked status of media server by nbemmcmd -listhosts -verbose:
MachineState = active for tape and disk jobs (14)
02-17-2015 11:32 PM
Are you working with the person who posted back in Dec?
Or are you having the same/similar issue?
02-17-2015 11:39 PM
Yes, Marianne...This issue still persists. Backup is getting successful but few channels are getting failed with below error
02/17/2015 04:10:38 - Error nbjm (pid=23658736) NBU status: 2027, EMM status: Media server is not active
Media server is not active (2027)
02-18-2015 12:04 AM
Show us output of nbemmcmd -listhosts at the time when error is seen.
It is possible that there is a network glitch at the time when the error occurs.
See this TN regarding EMM heartbeat timeout: http://www.symantec.com/business/support/index?page=content&id=TECH70827
(There seems to be a problem with Symantec Support site at the moment.)
My questions of 14 Jan were not answered:
Multiple interfaces on the media server?
vmoprcmd output shows drives in use on media-server.
Job details shows a problem with media-server-nb.
Curious to see output of 'nbemmcmd -listhosts -verbose'.
This will show us how 'media-server' is configured in EMM.
02-18-2015 12:39 AM
output of nbemmcmd -listhosts at present: But how to get the output for that perticular time when the issue occured?
XYZ-nb
ClusterName = ""
MachineName = "XYZ-nb"
FQName = "XYZ-nb"
LocalDriveSeed = ""
MachineDescription = ""
MachineFlags = 0xf7
MachineNbuType = media (1)
MachineState = active for tape and disk jobs (14)
MasterServerName = "ABC-nb"
NetBackupVersion = 7.5.0.4 (750400)
OperatingSystem = solaris (2)
ScanAbility = 5
02-18-2015 01:17 AM
My question again:
Multiple interfaces on the media server?
vmoprcmd output shows drives in use on media-server.
Job details shows a problem with media-server-nb.
How are media servers actually configured in NBU? With which name?
But how to get the output for that perticular time when the issue occured?
Pay someone to monitor backups...
Or do it yourself.
I have often in my level 1 and 2 Support days sat at a customer's site overnight to see what is happening at OS and network level when things go wrong in NBU.
Please read though the TN that I have posted to see which logs to check for EMM heartbeat errors.