cancel
Showing results for 
Search instead for 
Did you mean: 

vxdisksetup error

nmcherla
Level 3

Hi,

I am having problem initializing the disk with "vxdisksetup".  These are the LUNS which were removed from veritas & storage , and have been re-allocated back to the same system.  I can see the disks from OS and also in veritas but with "error" status. 

vxdisksetup exits with the following error, but I can see the device in both /dev/vx/dmp and also in /dev/vx/rdmp.  I tried removing the disk from veritas, remove the files from both /dev/vx/dmp and /dev/vx/rdmp, label the disk and then re-run "vxdctl enable" but seeing all the 3 disks in "error" status.  I didn't move the file disk.info file & run "vxconfigd -K".  Its a cluster node (Sun cluster) so wondering if there is anything I can do without moving disk.info file.

# vxdisk list |grep error
c2t5006048AD52F4106d14s2 auto            -            -            error
c2t5006048AD52F4106d35s2 auto            -            -            error
c2t5006048AD52F4106d36s2 auto            -            -            error

# vxdisksetup -i c2t5006048AD52F4106d14 format=sliced
prtvtoc: /dev/vx/rdmp/c2t5006048AD52F4106d14s2: No such device or address

# prtvtoc /dev/vx/rdmp/c2t5006048AD52F4106d14s2
prtvtoc: /dev/vx/rdmp/c2t5006048AD52F4106d14s2: No such device or address

# prtvtoc /dev/rdsk/c2t5006048AD52F4106d14s2
* /dev/rdsk/c2t5006048AD52F4106d14s2 partition map
*
* Dimensions:
*     512 bytes/sector
*     533 sectors/track
*      30 tracks/cylinder
*   15990 sectors/cylinder
*   65535 cylinders
*   65533 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
* Unallocated space:
*       First     Sector    Last
*       Sector     Count    Sector
*           0 1047872670 1047872669
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0 1047872670 1047872669
#
 

Question on vxdiskunsetup:

I have also noticed the following error on some disks, when running "vxdiskunsetup". 

#vxdiskunsetup c2t5006048AD52F4126d39

VxVM vxdiskunsetup ERROR V-5-2-3524 c2t5006048AD52F4126d39: No dmpnode is associated



Thanks,

10 REPLIES 10

Gaurav_S
Moderator
Moderator
   VIP    Certified

what about paths to those disks ? can you paste following:

vxdisk list c2t5006048AD52F4106d14s2

& similar output for other disks ...

you are getting prtvtoc output from raw disk or DMP node ?

Can you also paste:

# modinfo | grep -i vx

# vxdmpadm listctlr all

# vxdmpadm listenclosure all

# vxdisk -e list | egrep 'cxtxdcx | cxtxdx'   (put your errored disk devices here)

 

Gaurav

nmcherla
Level 3

Gaurav,

I am seeing error while  running prtvtoc for DMP node and don't have any issue for getting disk geometry for rdsk.  I have pasted the outputs also.

# prtvtoc /dev/vx/rdmp/c2t5006048AD52F4106d14s2
prtvtoc: /dev/vx/rdmp/c2t5006048AD52F4106d14s2: No such device or address

# vxdisk list |grep error
NONAMEs2     auto            -            -            error
c2t5006048AD52F4106d14s2 auto            -            -            error
c2t5006048AD52F4106d35s2 auto            -            -            error
c2t5006048AD52F4106d36s2 auto            -            -            error
#

# vxdisk list c2t5006048AD52F4106d14s2
Device:    c2t5006048AD52F4106d14s2
devicetag: c2t5006048AD52F4106d14
type:      auto
flags:     online error private autoconfig
pubpaths:  block=/dev/vx/dmp/c2t5006048AD52F4106d14s2 char=/dev/vx/rdmp/c2t5006048AD52F4106d14s2
guid:      -
udid:      INVALID
site:      -
Multipathing information:
numpaths:   1
emcpower18c     state=disabled
#
# vxdisk list c2t5006048AD52F4106d35s2
Device:    c2t5006048AD52F4106d35s2
devicetag: c2t5006048AD52F4106d35
type:      auto
flags:     online error private autoconfig
pubpaths:  block=/dev/vx/dmp/c2t5006048AD52F4106d35s2

char=/dev/vx/rdmp/c2t5006048AD52F4106d35s2
guid:      -
udid:      INVALID
site:      -
Multipathing information:
numpaths:   1
emcpower16c     state=disabled
#
# vxdisk list c2t5006048AD52F4106d36s2
Device:    c2t5006048AD52F4106d36s2
devicetag: c2t5006048AD52F4106d36
type:      auto
flags:     online error private autoconfig
pubpaths:  block=/dev/vx/dmp/c2t5006048AD52F4106d36s2 char=/dev/vx/rdmp/c2t5006048AD52F4106d36s2
guid:      -
udid:      INVALID
site:      -
Multipathing information:
numpaths:   1
emcpower15c     state=disabled
#

# modinfo|grep -i vx
 50 7bec6000  3e4e0 308   1  vxdmp (VxVM 5.0MP3: DMP Driver)
 52 7ba00000 209248 309   1  vxio (VxVM 5.0MP3 I/O driver)
 54 7bbe90f0    c78 310   1  vxspec (VxVM 5.0MP3 control/status driv)
244 7aa213c0    cb0 311   1  vxportal (VxFS 5.0_REV-5.0MP3A25_sol port)
245 79e00000 1d89e0  21   1  vxfs (VxFS 5.0_REV-5.0MP3A25_sol SunO)
263 79fe4000   a9e0 312   1  fdd (VxQIO 5.0_REV-5.0MP3A25_sol Qui)

# vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
emcp            PP_EMC          ENABLED      pp_emc0
c0              Disk            ENABLED      disk
c1              Disk            ENABLED      disk
 

# vxdmpadm listenclosure all

ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_
COUNT
================================================================================
===
disk              Disk           DISKS                CONNECTED    Disk        4
#

 # vxdisk -e list | egrep 'c2t5006048AD52F4106d14s2|c2t5006048AD52F4106d35s2|c2t5006048AD52F4106d36s2'
c2t5006048AD52F4106d14s2 auto      -             -            error        emcpower18c  -
c2t5006048AD52F4106d35s2 auto      -             -            error        emcpower16c  -
c2t5006048AD52F4106d36s2 auto      -             -            error        emcpower15c  -
 

# vxdmpadm getsubpaths ctlr=emcp|grep -i disabled
emcpower17c  DISABLED     -                       PP_EMC       pp_emc0        -
emcpower18c  DISABLED     -          c2t5006048AD52F4106d14s2 PP_EMC       pp_emc0        -
emcpower16c  DISABLED     -          c2t5006048AD52F4106d35s2 PP_EMC       pp_emc0        -
emcpower15c  DISABLED     -          c2t5006048AD52F4106d36s2 PP_EMC       pp_emc0        -
 

g_lee
Level 6

nmcherla,

You mentioned prtvtoc of the disk works - are you using the cXtXdX device or the emcpower devices?

Could you also provide the following output for the powerpath devices, since VxVM is using the emcpower devices as the underlying path for the disks that you've indicated are in error

# /etc/powermt display dev=emcpower15c
# /etc/powermt display dev=emcpower16c
# /etc/powermt display dev=emcpower17c
# /etc/powermt display dev=emcpower18c

nmcherla
Level 3

I am using cxtxdx device not the emcpower device for prtvtoc.  Powerpath doesn't list the emcpower devices that VxVM uses (emcpowerxxc). 

The Pseduo name that EMC powerpath uses for the device C2t5006048AD52F4106d14s2 is "emcpower54a",( VXVM uses emcpower18c. )  And both the paths are active and alive. As I said these are the LUNS removed and re-assigned to the same system and these devices were under veritas control before.  I think some how VxVM is using the previous information of the LUN as we can see the emcpower device used by Veritas is different from powerpath.
 

# vxdisk list c2t5006048AD52F4106d14s2
Device:    c2t5006048AD52F4106d14s2
devicetag: c2t5006048AD52F4106d14
type:      auto
flags:     online error private autoconfig
pubpaths:  block=/dev/vx/dmp/c2t5006048AD52F4106d14s2 char=/dev/vx/rdmp/c2t5006048AD52F4106d14s2
guid:      -
udid:      INVALID
site:      -
Multipathing information:
numpaths:   1
emcpower18c     state=disabled
 

#  powermt display dev=emcpower18c
Bad dev value emcpower18c, or not under Powerpath control.
#
 

# powermt display dev=c2t5006048AD52F4106d14s2
Pseudo name=emcpower54a
Symmetrix ID=000190102788
Logical device ID=0AFB
state=alive; policy=SymmOpt; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
3072 pci@3,700000/SUNW,emlxs@0/fp@0,0 c2t5006048AD52F4106d14s0 FA  7cA   active  alive      0      0
3073 pci@13,700000/SUNW,emlxs@0/fp@0,0 c4t5006048AD52F4109d14s0 FA 10cA   active  alive      0      0

 

# vxdisk list |grep error

DEVICE       TYPE            DISK         GROUP        STATUS
NONAMEs2     auto            -            -            error
c2t5006048AD52F4106d14s2 auto            -            -            error
c2t5006048AD52F4106d35s2 auto            -            -            error
c2t5006048AD52F4106d36s2 auto            -            -            error
 

# prtvtoc /dev/rdsk/c2t5006048AD52F4106d14s2
* /dev/rdsk/c2t5006048AD52F4106d14s2 partition map
*
* Dimensions:
*     512 bytes/sector
*     533 sectors/track
*      30 tracks/cylinder
*   15990 sectors/cylinder
*   65535 cylinders
*   65533 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
* Unallocated space:
*       First     Sector    Last
*       Sector     Count    Sector
*           0 1047872670 1047872669
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0 1047872670 1047872669
#
 

g_lee
Level 6

If the devices were used by Veritas before, then removed and represented without updating/removing the disks cleanly from Veritas first, then this would explain why the entries are showing in error state, as the configuration hasn't been updated cleanly.

At this point, the best bet is to:

1. remove the devices in VxVM:

# vxdisk rm <disk> ### ie: the disks shown in error

2. Remove the entries for those disks under /dev/vx/dmp and /dev/vx/rdmp

3. Rebuild the dmp device tree

# vxdctl initdmp

4. Now rescan

# vxdisk scandisks

 

NOTE: this still may not work as the devices were not removed cleanly, so the incorrect config may be stuck in the kernel. If the above does not work, I'd suggest scheduling a reboot rather than trying to remove the disk.info file, as this probably won't work either/may confuse the issue further.

To avoid this situation in future, ensure disks are removed from VxVM cleanly (ie: vxdisk rm, removing devices under /dev/vx/*dmp/, ensuring disks are not rescanned in after this point by stopping vxesd) PRIOR to cleaning up PowerPath and removing the disks from the OS.

nmcherla
Level 3

I have tried removing the disks (vxdisk rm), removing entries in /dev/vx/*dmp, and ran vxdctl enable.  But didn't run initdmp.  Its a prmiary cluster node (Sun cluster), will the initdmp by any chance failover the groups?

To remove the LUNS, do we need to remove the devices under /dev/vx/*dmp after removing the disk from veritas?

Thanks,

Gaurav_S
Moderator
Moderator
   VIP    Certified

initdmp should not ideally cause a failover of cluster....  but I doubt it to resolve the issue in your case.

I would reccomend to plan for outage  & do a reconfig reboot ... additionally it will be good if power devices are reconfigured too (powercf) which would be done during the boot time..

since devices were represented, I believe that power devices names were changed which veritas has not picked up.

I also noted above that EMC array is not listing in vxdmpadm listenclosure output... can you confirm if anything you have excluded ?

# vxdmpadm listexclude all

 

Gaurav

nmcherla
Level 3

Gaurav,

I did see that and nothing has been excluded.  What could be the reason?

# vxdmpadm listexclude all
# vxdmpadm listenclosure all
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT
===================================================================================
disk              Disk           DISKS                CONNECTED    Disk        4
#
 

Gaurav_S
Moderator
Moderator
   VIP    Certified

Again my recommendation would be to do a reconfig reboot ... not sure how it landed to this state...

btw, did you tried a devfsadm before running a vxdctl enable ?

 

Gaurav

nmcherla
Level 3

Yes.  I have used devfsadm to identify the new luns and then vxdctl enable.  array.info file have all the information but from where did the "vxdmpadm listctlr and  enclosure" getting the the information? I was trying to avoid reboot but looks like there is no other option for me at this point.  I will schedule a reboot for this server.  Thanks Gaurav and to every one who replied for my post.