cancel
Showing results for 
Search instead for 
Did you mean: 

Error: Drive path are down but media is available

ZXVII
Level 2

I am very new to NetBackUp. Could someone help me to troubleshoot this? I do not know where I should start.

Below results from vmoprcmd:

<NONE>

DRIVE STATUS

Drive Name Label Ready RecMID ExtMID Wr.Enbl. Type
Host DrivePath Status
=============================================================================
HP.ULTRIUM5-SCSI.000 No No No hcart3-Clean
milstrg01 {1,7,0,0} TLD

HP.ULTRIUM5-SCSI.001 No No No hcart3-Clean
milstrg01 {1,7,1,0} TLD

HP.ULTRIUM5-SCSI.002 No No No hcart3-Clean
milstrg01 {1,7,2,0} TLD

HP.ULTRIUM5-SCSI.003 No No No hcart3-Clean
milstrg01 {1,7,3,0} TLD

HP.ULTRIUM5-SCSI.004 No No No hcart3
milstrg04 {3,7,0,0} TLD

HP.ULTRIUM5-SCSI.005 No No No hcart3
milstrg04 {3,7,1,0} TLD

HP.ULTRIUM5-SCSI.006 No No No hcart3
milstrg04 {3,7,2,0} TLD

HP.ULTRIUM5-SCSI.007 No No No hcart3
milstrg04 {3,7,3,0} TLD

HP.ULTRIUM6-SCSI.000 No No No hcart3
milnbmaster nrst1a (MILINDNETAPP1A) SCAN-TLD
milnbmedia5230prd01 /dev/nst2 DOWN-TLD

HP.ULTRIUM6-SCSI.001 No No No hcart3
milnbmaster nrst0a (MILINDNETAPP1A) SCAN-TLD
milnbmedia5230prd01 /dev/nst0 DOWN-TLD

IBM.ULTRIUM-HH6.000 No No No hcart3-Clean
milnbmedia5230prd01 /dev/nst1 DOWN-TLD

IBM.ULTRIUM-HH6.001 No No No hcart3-Clean
milnbmedia5230prd01 /dev/nst3 DOWN-TLD

I try to UP drive and Reset the drive and it is up then back to DOWN-TLD again

Regards,

Jirawat

 

6 REPLIES 6

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The problem seems to be with this NBU Appliance :
milnbmedia5230prd01

All drives are down. 
Chances are slim that there is a problem with all of the drives. 
You need to double-check connectivity, zoning, etc. 

To check what the Appliance can see at OS level, you can run 'scan' and 'show' from the CLISH:
Main > Manage-> FiberChannel->Scan 
Main > Manage > FiberChannel > Show

Hi Marianne,

The result from Show as list below:

FC HBA card(s) are configured correctly.
**** FC HBA Cards ****
07:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
07:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
08:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
08:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
81:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
81:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
82:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
82:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

**** Drivers ****
qla2xxx is loaded
windrvr6 is not loaded

**** Ports ****
Bus ID Slot Port WWN Status Mode Speed Remote Ports
07:00.0 Slot5 21:00:00:24:FF:52:AC:9C Linkdown Initiator* 8 gbit/s
07:00.1 Slot5 21:00:00:24:FF:52:AC:9D Linkdown Initiator 8 gbit/s
08:00.0 Slot6 21:00:00:24:FF:52:AC:88 Online Initiator* 8 gbit/s 0x500308c38e12b00d
0x500308c38e12b005
08:00.1 Slot6 21:00:00:24:FF:52:AC:89 Online Initiator 8 gbit/s 0x500a098680601b4f
0x500a098690601b4f
81:00.0 Slot4 21:00:00:24:FF:52:AD:6A Online Initiator 8 gbit/s 0x500308c38e12b009
0x500308c38e12b001
81:00.1 Slot4 21:00:00:24:FF:52:AD:6B Online Initiator 8 gbit/s 0x500a098380601b4f
0x500a098390601b4f
82:00.0 Slot2 21:00:00:24:FF:52:AD:2A Linkdown Initiator 8 gbit/s
82:00.1 Slot2 21:00:00:24:FF:52:AD:2B Linkdown Initiator 8 gbit/s

*** Devices ****
Device Vendor ID Type Remote Port
/dev/sg10 NETAPP 495d4643552d4657 LUN 0x500a098690601b4f
/dev/sg11 NETAPP 495d4643552d4644 LUN 0x500a098690601b4f
/dev/sg12 NETAPP 545d4643546d7676 LUN 0x500a098690601b4f
/dev/sg13 NETAPP 495d4643552d462d LUN 0x500a098690601b4f
/dev/sg14 NETAPP 545d4643546d7678 LUN 0x500a098690601b4f
/dev/sg15 NETAPP 495d4643552d4642 LUN 0x500a098690601b4f
/dev/sg16 NETAPP 545d4643546d7731 LUN 0x500a098690601b4f
/dev/sg17 NETAPP 495d4643552d4666 LUN 0x500a098690601b4f
/dev/sg18 NETAPP 545d4643546d774e LUN 0x500a098690601b4f
/dev/sg19 NETAPP 545d4643546d7841 LUN 0x500a098690601b4f
/dev/sg20 NETAPP 495d4643552d4632 LUN 0x500a098680601b4f
/dev/sg21 NETAPP 545d4643546d7670 LUN 0x500a098680601b4f
/dev/sg22 NETAPP 495d4643552d4634 LUN 0x500a098680601b4f
/dev/sg23 NETAPP 545d4643546d7672 LUN 0x500a098680601b4f
/dev/sg24 NETAPP 495d4643552d4657 LUN 0x500a098680601b4f
/dev/sg25 NETAPP 495d4643552d4644 LUN 0x500a098680601b4f
/dev/sg26 NETAPP 545d4643546d7676 LUN 0x500a098680601b4f
/dev/sg27 NETAPP 495d4643552d462d LUN 0x500a098680601b4f
/dev/sg28 NETAPP 545d4643546d7678 LUN 0x500a098680601b4f
/dev/sg29 NETAPP 495d4643552d4642 LUN 0x500a098680601b4f
/dev/sg30 NETAPP 545d4643546d7731 LUN 0x500a098680601b4f
/dev/sg31 NETAPP 495d4643552d4666 LUN 0x500a098680601b4f
/dev/sg32 NETAPP 545d4643546d774e LUN 0x500a098680601b4f
/dev/sg33 NETAPP 545d4643546d7841 LUN 0x500a098680601b4f
/dev/sg34 HP Ultrium 6-SCSI 0x500308c38e12b001
/dev/sg35 QUANTUM Scalar i40-i80 0x500308c38e12b001
/dev/sg36 IBM ULTRIUM-HH6 0x500308c38e12b009
/dev/sg37 NETAPP 495d4643552d4632 LUN 0x500a098390601b4f
/dev/sg38 NETAPP 545d4643546d7670 LUN 0x500a098390601b4f
/dev/sg39 NETAPP 495d4643552d4634 LUN 0x500a098390601b4f
/dev/sg4 HP Ultrium 6-SCSI 0x500308c38e12b005
/dev/sg40 NETAPP 545d4643546d7672 LUN 0x500a098390601b4f
/dev/sg41 NETAPP 495d4643552d4657 LUN 0x500a098390601b4f
/dev/sg42 NETAPP 495d4643552d4644 LUN 0x500a098390601b4f
/dev/sg43 NETAPP 545d4643546d7676 LUN 0x500a098390601b4f
/dev/sg44 NETAPP 495d4643552d462d LUN 0x500a098390601b4f
/dev/sg45 NETAPP 545d4643546d7678 LUN 0x500a098390601b4f
/dev/sg46 NETAPP 495d4643552d4642 LUN 0x500a098390601b4f
/dev/sg47 NETAPP 545d4643546d7731 LUN 0x500a098390601b4f
/dev/sg48 NETAPP 495d4643552d4666 LUN 0x500a098390601b4f
/dev/sg49 NETAPP 545d4643546d774e LUN 0x500a098390601b4f
/dev/sg5 IBM ULTRIUM-HH6 0x500308c38e12b00d
/dev/sg50 NETAPP 545d4643546d7841 LUN 0x500a098390601b4f
/dev/sg51 NETAPP 495d4643552d4632 LUN 0x500a098380601b4f
/dev/sg52 NETAPP 545d4643546d7670 LUN 0x500a098380601b4f
/dev/sg53 NETAPP 495d4643552d4634 LUN 0x500a098380601b4f
/dev/sg54 NETAPP 545d4643546d7672 LUN 0x500a098380601b4f
/dev/sg55 NETAPP 495d4643552d4657 LUN 0x500a098380601b4f
/dev/sg56 NETAPP 495d4643552d4644 LUN 0x500a098380601b4f
/dev/sg57 NETAPP 545d4643546d7676 LUN 0x500a098380601b4f
/dev/sg58 NETAPP 495d4643552d462d LUN 0x500a098380601b4f
/dev/sg59 NETAPP 545d4643546d7678 LUN 0x500a098380601b4f
/dev/sg6 NETAPP 495d4643552d4632 LUN 0x500a098690601b4f
/dev/sg60 NETAPP 495d4643552d4642 LUN 0x500a098380601b4f
/dev/sg61 NETAPP 545d4643546d7731 LUN 0x500a098380601b4f
/dev/sg62 NETAPP 495d4643552d4666 LUN 0x500a098380601b4f
/dev/sg63 NETAPP 545d4643546d774e LUN 0x500a098380601b4f
/dev/sg64 NETAPP 545d4643546d7841 LUN 0x500a098380601b4f
/dev/sg7 NETAPP 545d4643546d7670 LUN 0x500a098690601b4f
/dev/sg8 NETAPP 495d4643552d4634 LUN 0x500a098690601b4f
/dev/sg9 NETAPP 545d4643546d7672 LUN 0x500a098690601b4f

*** Remote Appliances over FC ****
Please scan for remote appliances over FC first

*** Notes ****
(NOTE: Ports in mode "Initiator*" are configured for target mode when SAN Client FT Media
Server is active, however, are currently running in initiator mode, i.e. SAN Client
is disabled or inactive.)

What should I do next? 

Regards,

Jirawat

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

We can see the tape library and drives fine at Appliance OS level:

/dev/sg35 QUANTUM Scalar i40-i80 0x500308c38e12b001

/dev/sg34 HP Ultrium 6-SCSI 0x500308c38e12b001
/dev/sg36 IBM ULTRIUM-HH6 0x500308c38e12b009
dev/sg4 HP Ultrium 6-SCSI 0x500308c38e12b005
/dev/sg5 IBM ULTRIUM-HH6 0x500308c38e12b00d

To see why the drives are being DOWN'ed, search for the word DOWN in /var/log/messages on the appliance.

Next action will be based on the reason(s) for DOWN drives. 

Krutons
Moderator
Moderator
   VIP   

Incase you are unsure of how to elevate your access on the appliance to be able to access /var/log/messages check this link out.

https://www.veritas.com/content/support/en_US/doc/96220900-127024912-0/v113564108-127024912

Hi Marianne,

Thank you so much for your help. Unfortunately, there is nothing much on the /var/log/message. I look for the word DOWN as you told but it just displays as below result:

Aug 1 12:15:17 milnbmedia5230prd01 ltid[20263]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.001 (device 1)
Aug 1 12:35:17 milnbmedia5230prd01 ltid[68157]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 2)
Aug 1 20:15:09 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 2)
Aug 1 20:15:31 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.001 (device 3)
Aug 1 22:50:41 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.000 (device 0)
Aug 1 22:57:12 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.001 (device 1)
Aug 1 23:00:15 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.001 (device 3)
Aug 1 23:00:16 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 2)
Aug 1 23:00:17 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.000 (device 0)
Aug 1 23:40:21 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.001 (device 1)
Aug 1 23:45:18 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.000 (device 0)
Aug 1 23:46:47 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.001 (device 3)
Aug 1 23:51:46 milnbmedia5230prd01 ltid[21643]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 2)
Aug 2 03:47:12 milnbmedia5230prd01 ltid[77089]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 2)
Aug 2 05:47:16 milnbmedia5230prd01 ltid[77089]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.001 (device 3)
Aug 2 05:53:42 milnbmedia5230prd01 ltid[77089]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.000 (device 2)
Aug 2 06:00:16 milnbmedia5230prd01 ltid[77089]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.001 (device 1)
Aug 2 06:06:48 milnbmedia5230prd01 ltid[77089]: Operator/EMM server has DOWN'ed drive HP.ULTRIUM6-SCSI.000 (device 0)
Aug 2 06:13:17 milnbmedia5230prd01 ltid[77089]: Operator/EMM server has DOWN'ed drive IBM.ULTRIUM-HH6.001 (device 3)

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I was hoping to see more in messages file.

Can you check if there is VERBOSE entry in /usr/openv/volmgr/vm.conf ?
If not, please add this entry and restart NBU. 

With Media Manager running in verbose mode, all device-related actions are logged in messages file. 
We should be able to see what the device-related actions were that lead to drives being DOWN'ed. 
So, check the messages file to see what happened before drive DOWN line.

Please check bptm log as well for errors. 

Use robtest to confirm device config and that tapes can be loaded and unloaded: 
https://www.veritas.com/content/support/en_US/article.100022873.html

There are quite a number of reasons for drives being DOWNed, e.g. 
3 x I/O errors in 12 hours (https://www.veritas.com/support/en_US/article.100023335)
TapeAlerts (see NBU Admin Guide II - TapeAlert logs and codes) 
Incorrect mapping between robotic drive numbers and OS device names
Operators opening robot door and manually inserting tapes. When drive needs to unload, the robot finds the slot occupied and drive cannot unload
Various device-related issues : https://www.veritas.com/support/en_US/article.100014480.html

Old (still relevant) TN: https://www.veritas.com/support/en_US/article.100023264