cancel
Showing results for 
Search instead for 
Did you mean: 

SAN drives getting down in netbackup

Mehul_Vyas
Level 4

Hi Team,

We have Netbackup 8.1 on master. Currently the tape drives are getting down on one SAN media server and which is a shared drive. Need your help to resolve.

How we can identify if the drives are misconfigured and how to solve them back.

 

/usr/openv/volmgr/bin/vmoprcmd -d -devconfig -l -h ctooptmwdb1un2 -autoconfig -t
TPAC60 IBM 03592E08 47A9 0000078DCAB0 -1 -1 -1 -1 /dev/nst5 - -
TPAC60 IBM 03592E08 47A9 0000078DCAFE -1 -1 -1 -1 /dev/nst2 - -
TPAC60 IBM 03592E08 47A9 0000078DCAB8 -1 -1 -1 -1 /dev/nst7 - -
TPAC60 IBM 03592E08 47A9 0000078DCAF3 -1 -1 -1 -1 /dev/nst6 - -
TPAC60 IBM 03592E08 47A9 0000078DCB3C -1 -1 -1 -1 /dev/nst4 - -
TPAC60 IBM 03592E08 47A9 0000078DCB0C -1 -1 -1 -1 /dev/nst3 - -
TPAC60 IBM 03592E08 47A9 0000078DCB0A -1 -1 -1 -1 /dev/nst1 - -
TPAC60 IBM 03592E08 47A9 0000078DCAF5 -1 -1 -1 -1 /dev/nst0 - -

 

# /usr/openv/volmgr/bin/vmoprcmd -d ds -h ctooptmwdb1un2 -devconfig
TPC_DEV65 DRIVE OH1SF2C1R3 16 8 0 33 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst6 0000078DCAF3 - 32907 2 -1 -1 -1 -1 0 3720 0 0 - 0 i0000:40/0000:40:02.2/0000:47:00.0/host7/rport-7:0-4/target7:0:2/7:0:2:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 4 0 0
TPC_DEV65 DRIVE OH1SF2C1R4 16 8 0 34 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst5 0000078DCB0C - 128 1 -1 -1 -1 -1 0 2798 0 0 NEEDS~CLEANING 1 i0000:40/0000:40:02.2/0000:47:00.0/host7/rport-7:0-3/target7:0:1/7:0:1:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 5 0 0
TPC_DEV65 DRIVE OH1SF2C1R2 16 8 0 32 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst4 0000078DCAB0 - 128 1 -1 -1 -1 -1 0 1259 0 0 - 0 i0000:40/0000:40:02.2/0000:47:00.0/host7/rport-7:0-2/target7:0:0/7:0:0:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 6 0 0
TPC_DEV65 DRIVE OH1SF2C3R3 16 8 0 41 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst3 0000078DCAF5 - 128 1 -1 -1 -1 -1 0 3595 0 0 - 0 i0000:00/0000:00:03.0/0000:04:00.0/host5/rport-5:0-5/target5:0:3/5:0:3:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 11 0 0
TPC_DEV65 DRIVE OH1SF2C3R1 16 8 0 39 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst2 0000078DCB0A - 128 1 -1 -1 -1 -1 0 3504 0 0 - 0 i0000:00/0000:00:03.0/0000:04:00.0/host5/rport-5:0-4/target5:0:2/5:0:2:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 12 0 0
TPC_DEV65 DRIVE OH1SF2C3R4 16 8 0 42 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst1 0000078DCB3C - 128 1 -1 -1 -1 -1 0 2257 0 0 NEEDS~CLEANING 1 i0000:00/0000:00:03.0/0000:04:00.0/host5/rport-5:0-3/target5:0:1/5:0:1:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 13 0 0
TPC_DEV65 DRIVE OH1SF2C3R2 16 8 0 40 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst0 0000078DCAFE - 128 1 -1 -1 -1 -1 0 2255 0 0 NEEDS~CLEANING 1 i0000:00/0000:00:03.0/0000:04:00.0/host5/rport-5:0-2/target5:0:0/5:0:0:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 14 0 0
TPC_DEV65 DRIVE OH1SF2C1R1 16 8 0 46 - -1 -1 -1 -1 ctooptmwdb1un2 /dev/nst7 0000078DCAB8 - 32907 2 -1 -1 -1 -1 0 3565 0 0 - 0 i0000:40/0000:40:02.2/0000:47:00.0/host7/rport-7:0-5/target7:0:3/7:0:3:0 - IBM~~~~~03592E08~~~~~~~~47A9 - 15 0 0
TPC_DEV65 ROBOT ROBOT0 8 0 ctooptmwdb1un2 - - -1 -1 -1 -1 0000078AA6080402 crebm1300 crebm1300 IBM~~~~~03584L22~~~~~~~~1411 0 1 0

1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Without knowing what the rest of your environment looks like and how your drive names were selected/configured, probably safest to manually edit drive properties for each incorrect drive and specify correct path. 
There are only 6 to edit.
Restart NBU on the media server once completed. 

View solution in original post

11 REPLIES 11

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Have you tried to clean tape drives that show "NEEDS~CLEANING" message? 

Best to have cleaning cartridges in None pool defined as hcart-CLN (matching device config) to enable automatic cleaning. 

For further device troubleshooting, add VERBOSE entry to /usr/openv/volmgr/vm.conf on all media servers, followed by NBU restart.
Device-related errors (such as reason for drives being DOWN'ed) will be logged in /var/log/messages on the media server where the drive was DOWN'ed. 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Hello 

 

 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Could you please show us output of 
vmoprcmd -h ctooptmwdb1un2 -devconfig -l 
Seeing device config in more readable format will help to get a better view of NBU config.

You seem to be combining command options that don't really belong together... 

tpautoconf -t shows actual (correct) path for tape drive on the media server.
vmoprcmd -devconfig shows NBU config. 

It seems that NBU config for this media server is incorrect. 
Best to delete all devices for this media server and rerun device config. 
vmoprcmd -h <media-server-name> -timeout 900 -autoconfig  -a

Hello Marianne,

Below is the output of the command.

 

# ./vmoprcmd -h ctooptmwdb1un2 -devconfig -l
Device Robot Drive Robot Drive Device Second
Type Num Index Type DrNum Status Comment Name Path Device Path
robot 0 - TLD - - - - crebm1300
  drive - 4 hcart3 33 UP - OH1SF2C1R3 /dev/nst6
  drive - 5 hcart3 34 DOWN - OH1SF2C1R4 /dev/nst5
  drive - 6 hcart3 32 DOWN - OH1SF2C1R2 /dev/nst4
  drive - 11 hcart3 41 UP - OH1SF2C3R3 /dev/nst3
  drive - 12 hcart3 39 DOWN - OH1SF2C3R1 /dev/nst2
  drive - 13 hcart3 42 DOWN - OH1SF2C3R4 /dev/nst1
  drive - 14 hcart3 40 DOWN - OH1SF2C3R2 /dev/nst0
  drive - 15 hcart3 46 UP - OH1SF2C1R1 /dev/nst7

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I am honestly battling to compare all of the output - you had '-devconfig' in both commands in your 1st post:

vmoprcmd -d -devconfig -l -h ctooptmwdb1un2 -autoconfig -t
vmoprcmd -d ds -h ctooptmwdb1un2 -devconfig

To know for sure if there is a mismatch, I would compare output of '... autoconfig -t'  and   '... devconfig -l'  with  'scan -changer' output on robot control host crebm1300.

scan output on crebm1300 will give you serial numbers and drive position numbers.
'... autoconfig -t'  will give you serial numbers and device paths on the media server.
You can then use this info to confirm NBU device config.

Or just delete and use NBU wizard to reconfig  (... autoconfig -a).
Hopefully your drive name rules will re-add the tape drives names the same as you currently have.

Hello Marianne,

Please find below output.

Will it resolve the issue without deleting the drives and ran the device config wizard. Because when I ran the device config wizard by selecting this media server, I could see 3 tape drives status show as "Unconfigured"

[Prod root @ crebm1300 ~]

# vmoprcmd -h ctooptmwdb1un2 -autoconfig -t

TPAC60 IBM     03592E08        47A9 0000078DCAB0 -1 -1 -1 -1 /dev/nst5 - -

TPAC60 IBM     03592E08        47A9 0000078DCB3C -1 -1 -1 -1 /dev/nst4 - -

TPAC60 IBM     03592E08        47A9 0000078DCAF5 -1 -1 -1 -1 /dev/nst0 - -

TPAC60 IBM     03592E08        47A9 0000078DCAFE -1 -1 -1 -1 /dev/nst2 - -

TPAC60 IBM     03592E08        47A9 0000078DCB0A -1 -1 -1 -1 /dev/nst1 - -

TPAC60 IBM     03592E08        47A9 0000078DCB0C -1 -1 -1 -1 /dev/nst3 - -

TPAC60 IBM     03592E08        47A9 0000078DCAB8 -1 -1 -1 -1 /dev/nst7 - -

TPAC60 IBM     03592E08        47A9 0000078DCAF3 -1 -1 -1 -1 /dev/nst6 - -

[Prod root @ crebm1300 ~]

# vmoprcmd -h ctooptmwdb1un2 -devconfig -l

Device Robot Drive       Robot                    Drive       Device     Second

Type     Num Index  Type DrNum Status  Comment    Name        Path       Device Path

robot      0    -    TLD    -       -  -          -           crebm1300

  drive    -    4 hcart3   33      UP  -          OH1SF2C1R3  /dev/nst6

  drive    -    5 hcart3   34    DOWN  -          OH1SF2C1R4  /dev/nst5

  drive    -    6 hcart3   32    DOWN  -          OH1SF2C1R2  /dev/nst4

  drive    -   11 hcart3   41      UP  -          OH1SF2C3R3  /dev/nst3

  drive    -   12 hcart3   39    DOWN  -          OH1SF2C3R1  /dev/nst2

  drive    -   13 hcart3   42    DOWN  -          OH1SF2C3R4  /dev/nst1

  drive    -   14 hcart3   40    DOWN  -          OH1SF2C3R2  /dev/nst0

  drive    -   15 hcart3   46      UP  -          OH1SF2C1R1  /dev/nst7

[Prod root @ crebm1300 ~]

# scan -changer

************************************************************

*********************** SDT_CHANGER ************************

************************************************************

------------------------------------------------------------

Device Name  : "/dev/sg1024"

Passthru Name: "/dev/sg1024"

Volume Header: ""

Port: -1; Bus: -1; Target: -1; LUN: -1

Inquiry    : "IBM     03584L22        1411"

Vendor ID  : "IBM     "

Product ID : "03584L22        "

Product Rev: "1411"

Serial Number: "0000078AA6080402"

WWN          : ""

WWN Id Type  : 0

Device Identifier: "IBM     03584L22        0000078AA6080402"

Device Type    : SDT_CHANGER

NetBackup Robot Type: 8

Removable      : Yes

Device Supports: SCSI-3

Number of Drives : 46

Number of Slots  : 2500

Number of Media Access Ports: 255

Drive 1 Serial Number      : "0000078DCAF6"

Drive 2 Serial Number      : "0000078DCAB5"

Drive 3 Serial Number      : "0000078DCAF4"

Drive 4 Serial Number      : "0000078DCABB"

Drive 5 Serial Number      : "0000078DCB9B"

Drive 6 Serial Number      : "0000078DCBA0"

Drive 7 Serial Number      : "0000078DCBAA"

Drive 8 Serial Number      : "0000078DCBB2"

Drive 9 Serial Number      : "0000078DCBAC"

Drive 10 Serial Number      : "0000078DCBB5"

Drive 11 Serial Number      : "0000078DCBAD"

Drive 12 Serial Number      : "0000078DCBAE"

Drive 13 Serial Number      : "0000078DC751"

Drive 14 Serial Number      : "0000078DC6FA"

Drive 15 Serial Number      : "0000078DC68B"

Drive 16 Serial Number      : "0000078DC998"

Drive 17 Serial Number      : "0000078DC677"

Drive 18 Serial Number      : "0000078DC67B"

Drive 19 Serial Number      : "0000078DC682"

Drive 20 Serial Number      : "0000078DC652"

Drive 21 Serial Number      : "0000078DC69D"

Drive 22 Serial Number      : "0000078DC64A"

Drive 23 Serial Number      : "0000078DC678"

Drive 24 Serial Number      : "0000078DC6B4"

Drive 25 Serial Number      : "0000078DC689"

Drive 26 Serial Number      : "0000078DC691"

Drive 27 Serial Number      : "0000078DC651"

Drive 28 Serial Number      : "0000078DC982"

Drive 29 Serial Number      : "0000078DCAB2"

Drive 30 Serial Number      : "0000078DCADF"

Drive 31 Serial Number      : "0000078DCAF1"

Drive 32 Serial Number      : "0000078DCAB0"

Drive 33 Serial Number      : "0000078DCAF3"

Drive 34 Serial Number      : "0000078DCB0C"

Drive 35 Serial Number      : "0000078DCB0B"

Drive 36 Serial Number      : "0000078DCB49"

Drive 37 Serial Number      : "0000078DCB3E"

Drive 38 Serial Number      : "0000078DCAF2"

Drive 39 Serial Number      : "0000078DCB0A"

Drive 40 Serial Number      : "0000078DCAFE"

Drive 41 Serial Number      : "0000078DCAF5"

Drive 42 Serial Number      : "0000078DCB3C"

Drive 43 Serial Number      : "0000078D0571"

Drive 44 Serial Number      : "0000078D06EA"

Drive 45 Serial Number      : "0000078D044C"

Drive 46 Serial Number      : "0000078DCAB8"

Flags : 0x0

Reason: 0x0

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Your device mapping is wrong.

This is correct config as per device serial numbers in the robot :

Drive
position Serial number     Device path

32         0000078DCAB0   /dev/nst5
33         0000078DCAF3   /dev/nst6
34         0000078DCB0C   /dev/nst3 
39         0000078DCB0A   /dev/nst1
40         0000078DCAFE   /dev/nst2 
41         0000078DCAF5   /dev/nst0
42         0000078DCB3C   /dev/nst4
46         0000078DCAB8   /dev/nst7

 

Comparing with current config:

drive - 4  hcart3 33 UP   - OH1SF2C1R3 /dev/nst6      Correct
drive - 5  hcart3 34 DOWN - OH1SF2C1R4 /dev/nst5      must be nst3
drive - 6  hcart3 32 DOWN - OH1SF2C1R2 /dev/nst4      must be nst5
drive - 11 hcart3 41 UP   - OH1SF2C3R3 /dev/nst3      must be nst0
drive - 12 hcart3 39 DOWN - OH1SF2C3R1 /dev/nst2      must be nst1
drive - 13 hcart3 42 DOWN - OH1SF2C3R4 /dev/nst1      must be nst4
drive - 14 hcart3 40 DOWN - OH1SF2C3R2 /dev/nst0      must be nst2
drive - 15 hcart3 46 UP   - OH1SF2C1R1 /dev/nst7      Correct

Yes you are right Marianne.

Can you suggest what needs to be done to get this resolve.?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Without knowing what the rest of your environment looks like and how your drive names were selected/configured, probably safest to manually edit drive properties for each incorrect drive and specify correct path. 
There are only 6 to edit.
Restart NBU on the media server once completed. 

Thanks Marrianne,

Will try to change the path and refresh the nbu services on media server.