cancel
Showing results forΒ 
Search instead forΒ 
Did you mean:Β 

Vxdiskadm Replace disk does not show my failed disk

chuckchang23
Level 3

Hello all,

 

i would like to replace my failed disk via the vxdiskadm utility

 

EMMDPD04:/# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:sliced     rootdisk     rootdg       online
c0t1d0s2     auto:sliced     rootmirror   rootdg       online
c0t2d0s2     auto            -            -            error
c0t3d0s2     auto:none       -            -            online invalid
emcpower0s2  auto:sliced     bgw1dgemcpower0  bgw1dg       online shared
emcpower1s2  auto:sliced     ora1dgemcpower7  ora1dg       online shared
emcpower2s2  auto:sliced     fmm1dgemcpower6  fmm1dg       online shared
emcpower3s2  auto:sliced     bgw1dgemcpower1  bgw1dg       online shared
emcpower4s2  auto:cdsdisk    -            -            online
emcpower5s2  auto:cdsdisk    -            -            online
emcpower6s2  auto:sliced     lic1dgemcpower5  lic1dg       online shared
emcpower7s2  auto:cdsdisk    -            -            online
emcpower8s2  auto:sliced     bgw1dgemcpower8  bgw1dg       online shared
emcpower9s2  auto:sliced     bgw1dgemcpower9  bgw1dg       online shared
emcpower10s2 auto:sliced     bgw1dgemcpower10  bgw1dg       online shared
emcpower11s2 auto:sliced     bgw1dgemcpower11  bgw1dg       online shared
emcpower12s2 auto:sliced     bgw1dgemcpower12  bgw1dg       online shared
emcpower13s2 auto:sliced     bgw1dgemcpower13  bgw1dg       online shared
emcpower14s2 auto:sliced     bgw1dgemcpower14  bgw1dg       online shared
emcpower15s2 auto:sliced     bgw1dgemcpower15  bgw1dg       online shared
EMMDPD04:/# 
EMMDPD04:/# 
EMMDPD04:/# 
EMMDPD04:/# vxdiskadm
 
Volume Manager Support Operations
Menu: VolumeManager/Disk
 
 1      Add or initialize one or more disks
 2      Encapsulate one or more disks
 3      Remove a disk
 4      Remove a disk for replacement
 5      Replace a failed or removed disk
 6      Mirror volumes on a disk
 7      Move volumes from a disk
 8      Enable access to (import) a disk group
 9      Remove access to (deport) a disk group
 10     Enable (online) a disk device
 11     Disable (offline) a disk device
 12     Mark a disk as a spare for a disk group
 13     Turn off the spare flag on a disk
 14     Unrelocate subdisks back to a disk
 15     Exclude a disk from hot-relocation use
 16     Make a disk available for hot-relocation use
 17     Prevent multipathing/Suppress devices from VxVM's view
 18     Allow multipathing/Unsuppress devices from VxVM's view
 19     List currently suppressed/non-multipathed devices
 20     Change the disk naming scheme
 21     Get the newly connected/zoned disks in VxVM view
 22     Change/Display the default disk layouts
 23     Mark a disk as allocator-reserved for a disk group
 24     Turn off the allocator-reserved flag on a disk
 list   List disk information
 
 
 ?      Display help about menu
 ??     Display help about the menuing system
 q      Exit from menus
 
Select an operation to perform: 4
 
Remove a disk for replacement
Menu: VolumeManager/Disk/RemoveForReplace
  Use this menu operation to remove a physical disk from a disk
  group, while retaining the disk name.  This changes the state
  for the disk name to a "removed" disk.  If there are any
  initialized disks that are not part of a disk group, you will be
  given the option of using one of these disks as a replacement.
 
Enter disk name [<disk>,list,q,?] list
 
Disk group: rootdg
 
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
 
dm rootdisk     c0t0d0s2     auto     80321    583834230 -
dm rootmirror   c0t1d0s2     auto     80321    583850295 -
 
Disk group: bgw1dg
 
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
 
dm bgw1dgemcpower0 emcpower0s2 auto   96130    1312574752 -
dm bgw1dgemcpower1 emcpower3s2 auto   96130    1310486288 -
dm bgw1dgemcpower8 emcpower8s2 auto   96130    1576844000 -
dm bgw1dgemcpower9 emcpower9s2 auto   96130    1576844000 -
dm bgw1dgemcpower10 emcpower10s2 auto 96130    1576844000 -
dm bgw1dgemcpower11 emcpower11s2 auto 96130    1576844000 -
dm bgw1dgemcpower12 emcpower12s2 auto 96130    1314695328 -
dm bgw1dgemcpower13 emcpower13s2 auto 96130    1293746560 -
dm bgw1dgemcpower14 emcpower14s2 auto 96130    1576844000 -
dm bgw1dgemcpower15 emcpower15s2 auto 96130    1576844000 -
 
Disk group: fmm1dg
 
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
 
dm fmm1dgemcpower6 emcpower2s2 auto   80065    104711648 -
 
Disk group: lic1dg
 
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
 
dm lic1dgemcpower5 emcpower6s2 auto   67324    2019328  -
 
Disk group: ora1dg
 
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
 
dm ora1dgemcpower7 emcpower1s2 auto   80065    419280416 -
 
Enter disk name [<disk>,list,q,?] c0t2d0s2                
  VxVM  ERROR V-5-2-400
There is no disk named c0t2d0s2 in any disk group configuration.
  To get a list of disks enter "list".
 
Enter disk name [<disk>,list,q,?] q
 
 
Is this the proper way or is there any other way to replace my spare disk?
 
 
25 REPLIES 25

chuckchang23
Level 3

Hi KJBSS,

 

   How do i check what you mean by this 

       "Do you have this drive on an array that has active/passive support to your host? "

   Im not really that familiar with symantec 

  

kjbss
Level 5
Partner Accredited
chuckchang23 --

Here's an example of a disk device that is on an active/passive array:

Use vxdisk list on the device in order to see the paths to the device and whether they are all primary or not:

$ vxdisk list c1t5006016041E0B566d26s2
Device:    c1t5006016041E0B566d26s2
devicetag: c1t5006016041E0B566d26
type:      auto
hostid:   
disk:      name= id=1228990244.166.myhost81
group:     name=paytdg id=1228990226.158.myhost81
info:      format=cdsdisk,privoffset=256,pubslice=2,privslice=2
flags:     online ready private autoconfig
pubpaths:  block=/dev/vx/dmp/c1t5006016041E0B566d26s2 char=/dev/vx/rdmp/c1t5006016041E0B566d26s2
guid:      {f8ee5150-1dd1-11b2-a075-00144fb7bfa0}
udid:      DGC%5FRAID%205%5F6ARA600581%5F6006016077002100249BE954D575DD11
site:      -
version:   3.1
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=2 offset=65792 len=2031232 disk_offset=0
private:   slice=2 offset=256 len=65536 disk_offset=0
update:    time=1368695151 seqno=0.45
ssb:       actual_seqno=0.2
headers:   0 240
configs:   count=1 len=48144
logs:      count=1 len=7296
Defined regions:
config   priv 000048-000239[000192]: copy=01 offset=000000 enabled
config   priv 000256-048207[047952]: copy=01 offset=000192 enabled
log      priv 048208-055503[007296]: copy=01 offset=000000 enabled
lockrgn  priv 055504-055647[000144]: part=00 offset=000000
Multipathing information:
numpaths:   4
c1t5006016041E0B566d26s2        state=enabled   type=primary
c3t5006016141E0B566d26s2        state=enabled   type=primary
c3t5006016941E0B566d26s2        state=enabled   type=secondary
c1t5006016841E0B566d26s2        state=enabled   type=secondary

You can see above that some paths to the same LUN-device are "type=secondary" and others are "type=primary".  You cannot access LUN-devices via the secondary paths.  However, you should note that the "pubpaths:  block=/dev/vx/dmp/c1t5006016041E0B566d26s2 char=/dev/vx/rdmp/c1t5006016041E0B566d26s2" line shows you how you can realiably access the LUN no matter which paths to the LUN are currently "primary".  I:

$ prtvtoc /dev/vx/dmp/c1t5006016041E0B566d26s2
* /dev/vx/rdmp/c1t5006016041E0B566d26s2 partition map
*
* Dimensions:
*     512 bytes/sector
*      16 sectors/track
*       4 tracks/cylinder
*      64 sectors/cylinder
*   32768 cylinders
*   32766 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0   2097024   2097023
       7     15    01          0   2097024   2097023

Below you can see the error you can get if you try to access the device via a non-primary path:

$ prtvtoc /dev/dsk/c1t5006016841E0B566d26s2
prtvtoc: /dev/rdsk/c1t5006016841E0B566d26s2: Unable to read Disk geometry errno = 0x5

Below you can see that there is no error when you access the device via a primary path:

$ prtvtoc /dev/dsk/c1t5006016041E0B566d26s2
* /dev/dsk/c1t5006016041E0B566d26s2 partition map
*
* Dimensions:
*     512 bytes/sector
*      16 sectors/track
*       4 tracks/cylinder
*      64 sectors/cylinder
*   32768 cylinders
*   32766 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0   2097024   2097023
       7     15    01          0   2097024   2097023
 

Below you can see your controllers to your devices and their enclosure type:

$ vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c0              Disk            ENABLED      disk
c1              EMC_CLARiiON    ENABLED      emc_clariion0
c3              EMC_CLARiiON    ENABLED      emc_clariion0
c1              EMC_CLARiiON    ENABLED      emc_clariion1
c3              EMC_CLARiiON    ENABLED      emc_clariion1

EMC_CLARiiON arrays are "Active/Passiv" arrays, where only certain paths are active at a time.

You can use the "vxdmpadm listctlr all" command on your system to see what kind of arrays you have, if any, in your configuration.

Hope that helps, but I doubt it, because you have an I/O error, which is much more a hardware-physical layer problem.  You need your hardware looked at by your array's hardware Field Engineer (or possibly your server's hardware engineer if the problem is with the HBA in your host).

 

chuckchang23
Level 3

Hi Everyone, 

 

thank you for the answers :) its starting to make sense now. 

 

After all the research, is there a way to enable the path of this disk?

 

EMMDPD03:/var/VRTSvcs/lock# cat cvmvoldg4_ora1dg_vxnotify | tail
marked as failing path c0t2d0s2 belonging to dmpnode c0t2d0s2
enabled path c0t2d0s2 belonging to dmpnode c0t2d0s2
enabled dmpnode c0t2d0s2
disabled path c0t2d0s2 belonging to dmpnode c0t2d0s2
disabled dmpnode c0t2d0s2
marked as failing path c0t2d0s2 belonging to dmpnode c0t2d0s2
enabled path c0t2d0s2 belonging to dmpnode c0t2d0s2
enabled dmpnode c0t2d0s2
disabled path c0t2d0s2 belonging to dmpnode c0t2d0s2
disabled dmpnode c0t2d0s2
EMMDPD03:/var/VRTSvcs/lock# 

kjbss
Level 5
Partner Accredited

What was the resolution to your earlier I/O errors? 


If there are no known media nor SAN issues, Dynamic Multi Pathing (DMP) disables/enables disk paths after receiving SCSI Errors can be caused if your disk has no valid disk label on it (from TECH188277):

On the solaris server there were many disks with no disk label. DMP was unable to check the disk's associated paths because of there being no VTOC, so ended up failing the device and associated paths. Subsequently, DMP re-enables the path and disables it once again at the next check, and so on.


To enable a path:

vxdmpadm enable ctlr= enclosure=

...where the CTLR-NAME and ENCLR-NAME are displayed via the 'vxdmpadm listctlr all' command.


Make sure you label c0t2d0s2:

$ format c0t2d0
> label
> y
> q

 

All the while, monitor your OS and DMP logs for errors.  IE:

tail -f /var/adm/messages

tail -f /var/adm/vx/dmpevents.log

 

g_lee
Level 6

chuckchang23,

As others have pointed out (eg: kjbss, Gaurav), it appears there are problems with access / i/o to the disk - as you are unable to run prtvtoc at times but other times it works.

From the output provided, it appears the disk is local / not SAN or EMC powerdevice (as the other powerdevices are showing correctly, and vxdisk list shows it's an LSI disk).

Check that it appears correctly in format, if there are errors shown in iostat -En, the status in cfgadm.

To reenable the disk in vxvm, it needs to be accessible (ie: you need to sort out the i/o errors), and to have a valid label (ie: if you have intermittent problems running prtvtoc to display the label, then vxvm is going to show the disk in error as it won't be able to read the label either).

Also note per Yasuhisa's comment ( https://www-secure.symantec.com/connect/forums/vxdiskadm-replace-disk-does-not-show-my-failed-disk#comment-8727391 ) - it appears this disk has been used outside vxvm, as your prtvtoc shows s3 and s6 were previously mounted. So whatever is using the disk at the moment may have had its own reasons for removing/not having slice 2 -- so creating s2 now may cause issues.

If you want to use the disk in vxvm, you will need to relabel the disk; however unless you know the history of the machine / have confirmed whatever was using it previously is definitely not impacted by this, you should exercise caution / be prepared to potentially restore a backup of any data that was on this disk as the relabel may render the previous data inaccessible.

regards,

Grace

CliveE
Level 2
Employee

Hi chuckchang23,

Looks like you are making some progress thanks to the good replies here. Would be best to progress this type of enquiry over in the Storage Foundation forum as this not really a cluster server related command.

Cheers