Tape Library drive down

[Switcho]

Hi all,

As simple as that:

i've seen this error since one week:

when i right click on the drive and choose up drive, it come up but come down the next day.

where can i invistigate about the cause of that issue?

is it a hardware issue?

we have 3 servers, 2 media and one master, all running veritas netbackup 6.5.4,

thanks in advance.

[AAlmroth]

You could try to check the Problems report in the Admin Console and look for tape related entries.

You can also enable bptm logging on the media server, and watch for lines with TapeAlert.

In a majority of the cases you can from these see whether it is a media or drive related issue.

The TapeAlert indicates the type of issue, but each vendor may have specific event codes, so you may have to read up in the manual for the drive what the codes mean.

/A

[vinods]

Master server O.S. is windows You can check the event log for more information .

Then check the RSM service . Check sure it is not started .

[Switcho]

can you please explain how to do the bptm logging..

for the problems report in the admin console, kindly check the results below:

12/12/2010 2:44:48 PM sr1001bck002 sr1001csql1015 48322 Error error requesting media, TpErrno = Robot operation failed
12/12/2010 2:44:49 PM sr1001bck002 sr1001csql1015 48322 Warning media id 6566L4 load operation reported an error
12/12/2010 2:44:55 PM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
12/12/2010 2:46:09 PM sr1001bck002 sr1001csql1015 48322 Info begin writing backup id sr1001csql1015_1292150657, copy 1, fragment 1, to media id 6566L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 2:47:43 PM sr1001bck002 sr1001csql1015 48323 Info begin writing backup id sr1001csql1015_1292150832, copy 1, fragment 1, to media id 6566L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 2:49:02 PM sr1001bck002 sr1001csql1015 48324 Info begin writing backup id sr1001csql1015_1292150914, copy 1, fragment 1, to media id 6566L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 3:03:23 PM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
12/12/2010 7:00:19 PM sr1001bck002 SE1101APP001 48326 Error error requesting media, TpErrno = Robot operation failed
12/12/2010 7:00:20 PM sr1001bck002 SE1101APP001 48326 Warning media id 7949L4 load operation reported an error
12/12/2010 7:01:11 PM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
12/12/2010 7:01:55 PM sr1001bck002 sr1001ehb001 48328 Info begin writing backup id sr1001ehb001_1292166001, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 7:13:23 PM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
12/12/2010 7:13:24 PM sr1001bck002 sr1001mgt002 48329 Info begin writing backup id sr1001mgt002_1292166784, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 7:47:15 PM sr1001bck002 SE1101APP001 48326 Info begin writing backup id SE1101APP001_1292166000, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 7:55:23 PM sr1001bck002 sr1001agc001 48327 Info begin writing backup id sr1001agc001_1292169304, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 8:40:15 PM sr1001bck002 sr1001tsg001 48332 Info begin writing backup id sr1001tsg001_1292171995, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 8:42:11 PM sr1001bck002 sr1001spa001 48344 Info begin writing backup id sr1001spa001_1292172112, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 8:51:00 PM sr1001bck002 sr1001spw001 48346 Info begin writing backup id sr1001spw001_1292172639, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 9:01:57 PM sr1001bck002 sr1001sql002 48342 Info begin writing backup id sr1001sql002_1292173291, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 9:33:12 PM sr1001bck002 sr1001tsa001 48347 Info begin writing backup id sr1001tsa001_1292175173, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 10:02:53 PM sr1001bck002 sr1001tsa003 48348 Info begin writing backup id sr1001tsa003_1292176954, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 10:47:38 PM sr1001bck002 sr1001tsa004 48349 Info begin writing backup id sr1001tsa004_1292179637, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/12/2010 11:43:32 PM sr1001bck002 sr1001cmbx1001 48365 Info begin writing backup id sr1001cmbx1001_1292182985, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 12:55:35 AM sr1001bck002 sr1001cmbx1001 48368 Info begin writing backup id sr1001cmbx1001_1292187315, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 12:56:14 AM sr1001bck002 sr1001cmbx1001 48366 Info begin writing backup id sr1001cmbx1001_1292187352, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 2:25:15 AM sr1001bck002 sr1001cmbx1001 48367 Info begin writing backup id sr1001cmbx1001_1292192686, copy 1, fragment 1, to media id 6739L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 3:05:12 AM sr1001bck002 sr1001cmbx1001 48367 Info begin writing backup id sr1001cmbx1001_1292192686, copy 1, fragment 2, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 3:31:07 AM sr1001bck002 sr1001bes001 48363 Info begin writing backup id sr1001bes001_1292196649, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 3:38:36 AM sr1001bck002 sr1001cfil1010 48350 Info begin writing backup id sr1001cfil1010_1292197094, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 4:12:29 AM sr1001bck002 sr1001cfil1011 48351 Info begin writing backup id sr1001cfil1011_1292199128, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 4:36:17 AM sr1001bck002 SE1101APP001 48331 Info begin writing backup id SE1101APP001_1292200545, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 4:37:05 AM sr1001bck002 sr1001sql001 48341 Info begin writing backup id sr1001sql001_1292200598, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 5:15:38 AM sr1001bck002 sr1001sps001 48345 Info begin writing backup id sr1001sps001_1292202920, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 5:33:00 AM sr1001bck002 sr1001odb001 48352 Info begin writing backup id sr1001odb001_1292203967, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 9:22:56 AM sr1001bck002 sr1001sql010 48360 Error error requesting media, TpErrno = Robot operation failed
12/13/2010 9:22:57 AM sr1001bck002 sr1001sql010 48360 Warning media id 7949L4 load operation reported an error
12/13/2010 9:23:03 AM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
12/13/2010 9:33:23 AM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
12/13/2010 9:54:52 AM sr1001bck002 SE1101APP001 48380 Info begin writing backup id SE1101APP001_1292219662, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 9:56:03 AM sr1001bck002 SE1101APP001 48394 Info begin writing backup id SE1101APP001_1292219732, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 9:57:15 AM sr1001bck002 SE1101APP001 48395 Info begin writing backup id SE1101APP001_1292219804, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 9:58:29 AM sr1001bck002 SE1101APP001 48396 Info begin writing backup id SE1101APP001_1292219882, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 9:59:39 AM sr1001bck002 SE1101APP001 48397 Info begin writing backup id SE1101APP001_1292219949, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:00:49 AM sr1001bck002 SE1101APP001 48398 Info begin writing backup id SE1101APP001_1292220019, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:04:14 AM sr1001bck002 sr1001sql010 48399 Info begin writing backup id sr1001sql010_1292220223, copy 1, fragment 1, to media id 6735L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:12:49 AM sr1001bck002 sr1001bck001 48371 Info begin writing backup id sr1001bck001_1292220606, copy 1, fragment 1, to media id 6729L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:13:25 AM sr1001bck002 sr1001bck001 48400 Info begin writing backup id sr1001bck001_1292220792, copy 1, fragment 1, to media id 6729L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:17:38 AM sr1001bck002 sr1001csql1015 48373 Info begin writing backup id sr1001csql1015_1292220948, copy 1, fragment 1, to media id 6566L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:19:14 AM sr1001bck002 sr1001csql1015 48401 Info begin writing backup id sr1001csql1015_1292221123, copy 1, fragment 1, to media id 6566L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
12/13/2010 10:20:34 AM sr1001bck002 sr1001csql1015 48402 Info begin writing backup id sr1001csql1015_1292221203, copy 1, fragment 1, to media id 6566L4 on drive HP.ULTRIUM4-SCSI.000 (index 0)
 

[Switcho]

no events related to the tape library.

what is the RSM service? and why i have one drive that is working and the other isn't?

[Anantparashar]

Kindly check event id if any .

Run a robtest when there are no backups running and drive still shows down.

veritas\volmgr\bin\robtest

Choose the robot

Then try the command

s d (this will show you the available drive)

check the drive number which is down

run the command again

m s1 d1 (in this command i am moving a tape from slot 1 to drive 1 its just an example you can choose any slot but make sure you choose the correct drive)

m d1 s1 (moving it back)

if this command gives an error after execution  then its hardware problem most probably

Try to test the same on the working drive if that does not give any error that means its for sure that drive has some hardware problem

Thanx

Anant

[Anantparashar]

Make sure you run this command from the robot control host if in case the library is shared if its not then no issues.

Robot control host could be master or media server depending upon your configuration.

if you run this command from  a non robot control host machine it wont harm anything it will just give you an error no robots configured.

[Switcho]

one drive is working properly but the other gives that error:

Initiating MOVE_MEDIUM from address 1001 to 2
move_medium failed
sense key = 0x5, asc = 0x3b, ascq = 0xd, MEDIUM DESTINATION ELEMENT FULL

is that refers to a hardware issue?

[AAlmroth]

It would seem that you have a tape stuck in the tape drive. Open up the robot and eject the tape manually if the LCD panel does not allow you to eject the tape.

12/12/2010 7:13:23 PM sr1001bck002  0 Error error unloading media, TpErrno = Robot operation failed
 

/A

[AAlmroth]

You can enable this by creating the directory in <install path>\NetBackup\logs\bptm.

New bptm processes on the media server will start logging if the directory exists. You can increase the logging level by setting this in host properties->Media servers-<server>-Logging.

Maximum value 5 will give you a lot info, perhaps too much, but the mere existance of the directory should give you enough to troubleshoot.

/A

[USMAN_ALI]

Which OS you are using ?

This is not Netbackup issue...Usually the drive or the connection for the drive is the problem and that will have to be addressed. I have faced same issue. check with your hardware vendor.

Thanks,

Usman

[ayodeji]

this case looks like an hardware problem. youy can take a look at your library firmware and the device driver also.

[AAlmroth]

As I wrote previously, a tape seems to be stuck in the drive. Physically remove the tape. If not possible call hardware vendor for assistance.

/A