Down drives

[cyberninja]

Hello,

I want to UP the drives listed below. I have checked the library and it does not have any errors. NetBackup seams to be working fine. I found this because I was looking at the settings for the SUN SL150 tape library. We are getting ready to add a SL500 tape library and I wanted to compare.

<  tpconfig -d
Id  DriveName           Type   Residence
      Drive Path                                                       Status
****************************************************************************
0   HP.ULTRIUM5-SCSI.008 hcart3 TLD(1)  DRIVE=5
      /dev/rmt/4cbn                                                    UP
1   HP.ULTRIUM5-SCSI.009 hcart3 TLD(1)  DRIVE=1
      /dev/rmt/5cbn                                                    DOWN
2   HP.ULTRIUM5-SCSI.000 hcart3 TLD(1)  DRIVE=6
      /dev/rmt/2cbn                                                    UP
3   HP.ULTRIUM5-SCSI.011 hcart3 TLD(1)  DRIVE=4
      /dev/rmt/1cbn                                                    DOWN
4   HP.ULTRIUM5-SCSI.012 hcart3 TLD(1)  DRIVE=2
      /dev/rmt/0cbn                                                    UP
5   HP.ULTRIUM5-SCSI.013 hcart3 TLD(1)  DRIVE=3
      /dev/rmt/3cbn                                                    UP

Currently defined robotics are:
  TLD(1)     robotic path = /dev/sg/c0tw500104f000d1600al1

EMM Server = MasterServer

I think the tpconfig command can UP the drive but I was not able to get it to work. When I looked online all I could find was how to add and delete a drive with this command.  I was hoping to not delete the drive.

We are using NetBackup 7.5.0.7 on Solaris 10.

[Marianne]

Before you UP the drives, enable logging so that the reason will be logged in /var/adm/messages if drives should go DOWN again.

Add VERBOSE to /usr/openv/volmr/vm.conf and restart NBU.

To UP a drive, use vmoprcmd command:

vmoprcmd -up 1

vmoprcmd -up 4

[cyberninja]

Thanks for giving me a hand. 

The /usr/openv/volmr/vm.conf only exists on the master server. The media server that has the tape library conneted to it doesn't have the file. I add the line to the Master server's file. I have to wait for the backup jobs to finish beflre I can restart NetBackup though. 

I ran the vmoprcmd -up command on both drives. Drive 1 came up but drive 2 didn't. I tried vmoprcmd -up 3 instead of vmoprcmd -up 4 and it came backup. 

I will check the servers again in the morning. 

[Marianne]

vm.conf needs to be on each media server. Especially on the media server where drives are DOWN.

If the file does not exist, it must be created. 

When NBU is restarted after adding VERBOSE entry to vm.conf, you will see that media manager processes run with '-v' option.
This will ensure that these processes now log actions and errors to /var/adm/messages.

Check drive status with vmoprcmd -d, rather than tpconfig.

[cyberninja]

How should I create the vm.conf file? I just copied the one on the master server to the media server.

This is what the Master servers vm.conf has in it. Am I missing anything.

VERBOSE
MEDIA_ID_BARCODE_CHARS = 0 8 1:2:3:4:5:6
MEDIA_ID_BARCODE_CHARS = 1 8 1:2:3:4:5:6
 

I ran the vmoprcmd -d and got this output.

< vmoprcmd -d

                                PENDING REQUESTS

                                     <NONE>

                                  DRIVE STATUS

Drv Type   Control  User      Label  RecMID  ExtMID  Ready   Wr.Enbl.  ReqId
  0 hcart3   TLD                -                     No       -         0  
  1 hcart3   TLD                -                     No       -         0  
  2 hcart3   TLD                -                     No       -         0  
  3 hcart3   TLD                -                     No       -         0  
  4 hcart3   TLD                -                     No       -         0  
  5 hcart3   TLD                -                     No       -         0  

                             ADDITIONAL DRIVE STATUS

Drv DriveName            Shared    Assigned        Comment                   
  0 HP.ULTRIUM5-SCSI.008  Yes      -                                         
  1 HP.ULTRIUM5-SCSI.009  Yes      -                                         
  2 HP.ULTRIUM5-SCSI.000  Yes      -                                         
  3 HP.ULTRIUM5-SCSI.011  Yes      -                                         
  4 HP.ULTRIUM5-SCSI.012  Yes      -                                         
  5 HP.ULTRIUM5-SCSI.013  Yes      -           

---

I had an issue where the the drives where not showing when I rebooted NetBackup. Using the vmoprcmd command they where still showing up with the tpconfig command though. I removed the last two lines in the vm.conf file and rebooted NetBackup and I was able to see the drives again. 

[Marianne]

Your drives are shared. Above vmoprcmd output shows all drives UP and under TLD robot control on this particular server.

To know status on other media servers, run 'vmoprcmd -d -h <media-server>' for each media server.

vm.conf with VERBOSE entry needs to be on each media server to troubleshoot DOWN drives.

The media server that has a problem with a tape drive will DOWN the drive. So, this is where verbose media manager processes will log hardware issues to its own messages file.

The MEDIA_ID_BARCODE_CHARS should only be in vm.conf of robot control host and where the GUI is used to run inventory (e.g. master).

[sanjaynaidu]

HAve u have checked the dwn drive status from OS level as well.

Please compare the 2 outputs tpconfig -d and tpautoconf -t and check whether u can see the drive in OS level. before proccedings furthur.

[cyberninja]

Thanks for the info.

I will check the other media servers and make sure that the vm.conf file exists.

Drive 3 went down over the weekend. I havepasted the log (/var/adm/messages), where it shows that NetBackup downed the drive. I'm looking at the NetBckup logs now but I don't see anything yet.

Sep  5 18:57:15 nbumedia02 tldcd[6327]: [ID 277581 daemon.notice] inquiry() function processing library STK      SL150            0225:
Sep  5 18:57:15 SolarisMedia02 tldcd[6327]: [ID 876984 daemon.notice] TLD(1) initiating MOVE_MEDIUM from addr 1040 to addr 504
Sep  5 18:57:33 SolarisMedia02 tldd[8322]: [ID 436183 daemon.notice] DecodeDismount: TLD(1) drive 4, Actual status: Unable to SCSI unload drive
Sep  5 18:57:36 SolarisMedia02 ltid[8291]: [ID 527589 daemon.notice] LTID - Sent ROBOTIC request, Type=1, Param2=0
Sep  5 18:57:36 SolarisMedia02 tldd[8322]: [ID 902248 daemon.notice] TLD(1) MountTape U00876 on drive 1, from slot 44
Sep  5 18:57:36 SolarisMedia02 tldcd[8371]: [ID 676456 daemon.info] tldcd.c.3021, process_request(), received command=1, from peername=SolarisMedia02, version 50
Sep  5 18:57:36 SolarisMedia02 tldcd[8371]: [ID 109388 daemon.notice] Processing MOUNT, TLD(1) drive 1, slot 44, barcode U00876          , vsn U00876
Sep  5 18:57:36 SolarisMedia02 tldcd[6337]: [ID 978419 daemon.notice] TLD(1) opening robotic path /dev/sg/c0tw500104f000d1600al1
Sep  5 18:57:37 SolarisMedia02 ltid[8291]: [ID 780347 daemon.error] Operator/EMM server has DOWN'ed drive HP.ULTRIUM5-SCSI.011 (device 3)
Sep  5 18:58:09 SolarisMedia02 tldcd[6327]: [ID 559681 daemon.notice] TLD(1) closing/unlocking robotic path
Sep  5 18:58:09 SolarisMedia02 tldcd[6337]: [ID 277581 daemon.notice] inquiry() function processing library STK      SL150            0225:

[cyberninja]

Thanks for repling to my post. I can see the drive with the OS

[mph999]

Was the drive in use when it went down ?  If so, bptm log  or messages should give some indication as to why.

[Marianne]

DecodeDismount: TLD(1) drive 4, Actual status: Unable to SCSI unload drive

This is the reason for drive being downed.

TLD(1) drive 4 is the drive that was DOWNed. Output from tpconfig: Index 3, Drive name HP.ULTRIUM5-SCSI.011

3   HP.ULTRIUM5-SCSI.011 hcart3 TLD(1)  DRIVE=4
      /dev/rmt/1cbn                                                    DOWN

Operator/EMM server has DOWN'ed drive HP.ULTRIUM5-SCSI.011 (device 3)

Please check if there is a stuck tape in the drive.

Incorrect device mapping will also lead to tape drive dismount failure.

[cyberninja]

Looking at the SL150 web interface, I don't see any tapes in any of the 6 drives. As for drive mapping, last time I looked at the WWPN they matched up with the switch. I willouble check.

[Marianne]

Sometimes best to physically check... Or use robtest.

Device mapping is about the OS device name that need to correspond with the drive position in the robot.

So, compare 'scan -changer' output on the robot control host with these commands on each media server:

tpconfig -l

tpautoconf -t

[cyberninja]

The drive has not come down again.

I looked at the output of the commands you told me to run and I didn't see anything that looked wrong. I have pasted it below for your review.

< sudo scan -changer
************************************************************
*********************** SDT_CHANGER ************************
************************************************************
------------------------------------------------------------
Device Name  : "/dev/sg/c0tw500104f000d1600al1"
Passthru Name: "/dev/sg/c0tw500104f000d1600al1"
Volume Header: ""
Port: -1; Bus: -1; Target: -1; LUN: -1
Inquiry    : "STK     SL150           0225"
Vendor ID  : "STK     "
Product ID : "SL150           "
Product Rev: "0225"
Serial Number: "464970G+1336SY1459"
WWN          : ""
WWN Id Type  : 0
Device Identifier: "STK     SL150           464970G+1336SY1459"
Device Type    : SDT_CHANGER
NetBackup Robot Type: 8
Removable      : Yes
Device Supports: SCSI-5
Number of Drives : 8
Number of Slots  : 119
Number of Media Access Ports: 4
Drive 1 Serial Number      : "HU1331WR4P"
Drive 2 Serial Number      : "HU1331WR6K"
Drive 3 Serial Number      : "HU1331WR65"
Drive 4 Serial Number      : "HU1331WR5A"
Drive 5 Serial Number      : "HU1331WR67"
Drive 6 Serial Number      : "HU13520KRJ"
Drive 7 Serial Number      : ""
Drive 8 Serial Number      : ""
Flags : 0x0
Reason: 0x0

< sudo tpconfig -l
Device Robot Drive       Robot                    Drive                 Device         Second
Type     Num Index  Type DrNum Status  Comment    Name                  Path           Device Path
robot      1    -    TLD    -       -  -          -                     /dev/sg/c0tw500104f000d1600al1
  drive    -    0 hcart3    5      UP  -          HP.ULTRIUM5-SCSI.008  /dev/rmt/4cbn
  drive    -    1 hcart3    1      UP  -          HP.ULTRIUM5-SCSI.009  /dev/rmt/5cbn
  drive    -    2 hcart3    6      UP  -          HP.ULTRIUM5-SCSI.000  /dev/rmt/2cbn
  drive    -    3 hcart3    4      UP  -          HP.ULTRIUM5-SCSI.011  /dev/rmt/1cbn
  drive    -    4 hcart3    2      UP  -          HP.ULTRIUM5-SCSI.012  /dev/rmt/0cbn
  drive    -    5 hcart3    3      UP  -          HP.ULTRIUM5-SCSI.013  /dev/rmt/3cbn

< sudo tpautoconf -t
TPAC60 HP      Ultrium 5-SCSI  Y5BS HU1331WR67 -1 -1 -1 -1 /dev/rmt/4cbn - -
TPAC60 HP      Ultrium 5-SCSI  Y5BS HU1331WR4P -1 -1 -1 -1 /dev/rmt/5cbn - -
TPAC60 HP      Ultrium 5-SCSI  Y5BS HU13520KRJ -1 -1 -1 -1 /dev/rmt/2cbn - -
TPAC60 HP      Ultrium 5-SCSI  Y5BS HU1331WR5A -1 -1 -1 -1 /dev/rmt/1cbn - -
TPAC60 HP      Ultrium 5-SCSI  Y5BS HU1331WR6K -1 -1 -1 -1 /dev/rmt/0cbn - -
TPAC60 HP      Ultrium 5-SCSI  Y5BS HU1331WR65 -1 -1 -1 -1 /dev/rmt/3cbn - -

[Marianne · Accepted solution]

The problem is not with device mapping. 
Drive 4 mapping is correct:

Drive 4 Serial Number      : "HU1331WR5A"

 drive    -    3 hcart3    4      UP  -          HP.ULTRIUM5-SCSI.011  /dev/rmt/1cbn

TPAC60 HP      Ultrium 5-SCSI  Y5BS HU1331WR5A -1 -1 -1 -1 /dev/rmt/1cbn - -

You will need to check what is wrong and why tape cannot be dismounted.

If you do not have access to the tape library, try robtest: http://www.symantec.com/docs/TECH83129 

start with 's d'  to see what is in the drives.