cancel
Showing results for 
Search instead for 
Did you mean: 

lun over 2TB got error status from vxdisk list

maxpaynebupt
Level 3

Hi,

 

I have tried to bring up this lun for 2 days, and got nearly no progress, sometimes the lun can be online, but if I try to use vxdisksetup to setup it, it would become error again, I am really despairing, I hope someone can help me to solve this, thank u.

 

HBWHCG01:/# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
emcpower0s2  auto            -            -            error
emcpower1s2  auto:none       -            -            online invalid
emcpower2s2  auto:none       -            -            online invalid
emcpower3s2  auto:none       -            -            online invalid
emcpower4s2  auto:none       -            -            online invalid
emcpower5s2  auto:none       -            -            online invalid
emcpower6s2  auto:none       -            -            online invalid

HBWHCG01:/# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 36348 alt 2 hd 255 sec 126>
          /pci@0,0/pci8086,3408@1/pci1000,9262@0/sd@0,0
       1. c1t0d0 <DGC-RAID 10-0223-2.32TB>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,0
       2. c1t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,1
       3. c1t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,2
       4. c1t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,3
       5. c1t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,4
       6. c1t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,5
       7. c1t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,6
       8. c2t0d0 <DGC-RAID 10-0223-2.32TB>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,0
       9. c2t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,1
      10. c2t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,2
      11. c2t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,3
      12. c2t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,4
      13. c2t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,5
      14. c2t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,6
      15. emcpower0a <DGC-RAID 10-0223-2.32TB>
          /pseudo/emcp@0
      16. emcpower1a <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pseudo/emcp@1
      17. emcpower2a <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pseudo/emcp@2
      18. emcpower3a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@3
      19. emcpower4a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@4
      20. emcpower5a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@5
      21. emcpower6a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@6
Specify disk (enter its number): 15
selecting emcpower0a
[disk formatted]


FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        fdisk      - run the fdisk program
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        inquiry    - show vendor, product and revision
        volname    - set 8-character volume name
        !<cmd>     - execute <cmd>, then return
        quit
format> p


PARTITION MENU:
        0      - change `0' partition
        1      - change `1' partition
        2      - change `2' partition
        3      - change `3' partition
        4      - change `4' partition
        5      - change `5' partition
        6      - change `6' partition
        expand - expand label to use whole disk
        select - select a predefined table
        modify - modify a predefined partition table
        name   - name the current table
        print  - display the current table
        label  - write partition map and label to the disk
        !<cmd> - execute <cmd>, then return
        quit
partition> p
Current partition table (original):
Total disk sectors available: 4991205341 + 16384 (reserved sectors)

Part      Tag    Flag     First Sector          Size          Last Sector
  0        usr    wm                34         2.32TB           4991205341    
  1 unassigned    wm                 0            0                0    
  2 unassigned    wm                 0            0                0    
  3 unassigned    wm                 0            0                0    
  4 unassigned    wm                 0            0                0    
  5 unassigned    wm                 0            0                0    
  6 unassigned    wm                 0            0                0    
  8   reserved    wm        4991205342         8.00MB           4991221725    
 

1 ACCEPTED SOLUTION

Accepted Solutions

g_lee
Level 6

maxpaynebupt:

The bigger issue appears to be that all paths to the disk are showing dead in powermt display - so the OS & powerpath cannot see this device properly, so VxVM can't see it until that is fixed.

HBWHCG01:/# /etc/powermt display dev=emcpower0a
Pseudo name=emcpower0a
CLARiiON ID=FCNMM124800004 [MMSUN]
Logical device ID=6006016068EC3200C4B6FD8ADB9BE211
state=dead; policy=CLAROpt; priority=0; queued-IOs=0;
Owner: default=Unknown, current=Unknown Array failover mode: 1
==============================================================================
--------------- Host ---------------   - Stor -   -- I/O Path --  -- Stats ---
###  HW Path               I/O Paths    Interf.   Mode    State   Q-IOs Errors
==============================================================================
3072 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0 c1t0d0s0  SP A1     active  dead        0      0
3073 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0 c2t0d0s0  SP B1     active  dead        0      0

Is emcpower0 definitely the correct disk? There are no devices in the latest format output for emcpower0 or c[12]t0d0 - so the OS isn't seeing the disk either.

Have you tried cleaning the OS device tree?

# devfsadm -Csv   #### -s doesn't perform the clean, but you can see if will do anything

If you're sure this is the correct disk:

# devfsadm -Cv ### clean up

# /etc/powermt check  ### if devices are dead it will prompt you to remove them

Then check the array / server to ensure the disks show up correctly in format, and then try to rescan OS -> powerpath -> vxdisk again

 

EDIT: Additional check to see if using persistent naming:

# vxddladm get namingscheme

Note this still won't fix/affect anything until the OS & powerpath see the disk properly, so you still need to fix that first before attempting to rescan VxVM

View solution in original post

12 REPLIES 12

maxpaynebupt
Level 3

And , it's a standalone server with a EMC ax4-5 disk array connected by two fc fibre.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

You did not mention your SF version?

Try this procedure:

 

First, format the lun using the /dev/vx/rdmp name, and set the size of partition 0 to 0tb, and label the disk.

# format -e /dev/vx/rdmp/emcpower#

Run vxdisksetup against the disk.

# /etc/vx/bin/vxdisksetup -i -f emcpower# format=sliced

Run vxdisk init.

# vxdisk -f init emcpower# format=sliced

Then create a dg using that disk as the first disk.

# vxdg init test_dg cds=off testd1=emcpower#

 

maxpaynebupt
Level 3

thanks for your reply^_^

 

Is a 2.3t lun too big for vx? Do I need to resize it? 

I already tried to resize the lun to 125GB, but stuck at the system can not recognize the new lun........

Ok, I will try to resize the lun back to 2380GB, and try it again, but the rebound of the emc would take about 6 hours, I will come back after that, thank u.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Please let us know which SF version?

pkginfo -l VxVM

 

maxpaynebupt
Level 3

It seems the vxVM is still trying to find the old lun:

 

Untitled.png

maxpaynebupt
Level 3

HBWHCG01:/# pkginfo -l vxvm
ERROR: information for "vxvm" was not found
HBWHCG01:/# pkginfo -l VRTSvxvm
   PKGINST:  VRTSvxvm
      NAME:  Binaries for VERITAS Volume Manager by Symantec
  CATEGORY:  system
      ARCH:  i386
   VERSION:  5.1,REV=10.06.2009.21.48
   BASEDIR:  /
    VENDOR:  Symantec Corporation
      DESC:  Virtual Disk Subsystem
    PSTAMP:  5.1.102.000-5.1SP1RP2_x86-2011-09-13
  INSTDATE:  Apr 02 2012 14:51
   HOTLINE:  http://support.veritas.com/phonesup/phonesup_ddProduct_.htm
     EMAIL:  support@veritas.com
    STATUS:  completely installed
     FILES:      875 installed pathnames
                  44 shared pathnames
                 112 directories
                 402 executables
              375553 blocks used (approx)

HBWHCG01:/#

g_lee
Level 6

What version of Powerpath are you using?

# pkginfo -l EMCpower

Try the following to remove/rescan to pick up the disk:

# ps -ef |grep vxesd

#### if vxesd is running, stop it while you clean up the config

# vxddladm stop eventsource

#### now remove the disk:

# vxdisk rm emcpower0s2

# vxdisk list  ### check emcpower0 is no longer listed

#### after you resize the lun, rescan OS to pick up

# cfgadm -al

# cfgadm -c configure c#

### rescan powerpath

# /etc/powermt config

# /etc/powercf config

# /etc/powercf -q

# /etc/powermt save

#### check the following outputs:

# /etc/powermt display dev=emcpower0a

# prtvtoc /dev/rdsk/emcpower0a

# ls -la /dev/*dsk/emcpower0*   ### check powerdevices created correctly as this can be issue in old versions of powerpath

#### If prtvtoc shows the correct size:

# vxdisk scandisks

# vxdisk list emcpower0s2

maxpaynebupt
Level 3

Hi,

 

It seems to not work, there should be some database of vxdisk store the lun info and prevent the system from regain the expanded new lun.

 

 

HBWHCG01:/# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 36348 alt 2 hd 255 sec 126>
          /pci@0,0/pci8086,3408@1/pci1000,9262@0/sd@0,0
       1. c1t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,1
       2. c1t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,2
       3. c1t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,3
       4. c1t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,4
       5. c1t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,5
       6. c1t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,6
       7. c2t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,1
       8. c2t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,2
       9. c2t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,3
      10. c2t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,4
      11. c2t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,5
      12. c2t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,6
      13. emcpower1a <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pseudo/emcp@1
      14. emcpower2a <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pseudo/emcp@2
      15. emcpower3a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@3
      16. emcpower4a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@4
      17. emcpower5a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@5
      18. emcpower6a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@6
Specify disk (enter its number):  ^C
HBWHCG01:/#
HBWHCG01:/#
HBWHCG01:/#  pkginfo -l EMCpower
   PKGINST:  EMCpower
      NAME:  EMC PowerPath
  CATEGORY:  system
      ARCH:  i386
   VERSION:  5.3.P03_b003
   BASEDIR:  /opt
    VENDOR:  EMC Corporation
      DESC:  EMC Powerpath Multipathing, version 5.3.P03_b003
    PSTAMP:  dev:2011-12-27T07:23:25EST
  INSTDATE:  Apr 04 2013 04:24
    STATUS:  completely installed
     FILES:      148 installed pathnames
                  34 directories
                  67 executables
               68203 blocks used (approx)

HBWHCG01:/# ps -ef |grep vxesd
    root   814     1   0 22:16:24 ?           0:00 /sbin/vxesd -x dmp_monitor_osevent=off
    root  4243  4104   0 22:17:35 pts/2       0:00 grep vxesd
HBWHCG01:/#
HBWHCG01:/#  vxddladm stop eventsource
HBWHCG01:/# vxdisk rm emcpower0s2
VxVM vxdisk ERROR V-5-1-639 Failed to obtain locks:
        emcpower0s2: no such object in the configuration
HBWHCG01:/# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
emcpower1s2  auto:none       -            -            online invalid
emcpower2s2  auto:none       -            -            online invalid
emcpower3s2  auto:none       -            -            online invalid
emcpower4s2  auto:none       -            -            online invalid
emcpower5s2  auto:none       -            -            online invalid
emcpower6s2  auto:none       -            -            online invalid
HBWHCG01:/#  cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
Slot1                          pci-pci/hp   connected    configured   ok
Slot2                          pci-pci/hp   connected    configured   ok
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 disk         connected    configured   unknown
c1                             fc-private   connected    configured   unknown
c1::500601613ba07373           disk         connected    configured   unknown
c2                             fc-private   connected    configured   unknown
c2::500601693ba07373           disk         connected    configured   unknown
pcie3                          unknown      empty        unconfigured unknown
pcie4                          unknown      empty        unconfigured unknown
pcieb5.pcie0                   etherne/hp   connected    configured   ok
sata0/0                        sata-port    empty        unconfigured ok
sata0/1                        sata-port    empty        unconfigured ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
sata0/4                        sata-port    empty        unconfigured ok
sata0/5                        sata-port    empty        unconfigured ok
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok
usb1/1                         unknown      empty        unconfigured ok
usb1/2                         unknown      empty        unconfigured ok
usb2/1                         usb-device   connected    configured   ok
usb2/2                         unknown      empty        unconfigured ok
usb3/1                         unknown      empty        unconfigured ok
usb3/2                         unknown      empty        unconfigured ok
usb3/3                         usb-hub      connected    configured   ok
usb3/3.1                       unknown      empty        unconfigured ok
usb3/3.2                       unknown      empty        unconfigured ok
usb3/3.3                       unknown      empty        unconfigured ok
usb3/4                         unknown      empty        unconfigured ok
usb3/5                         unknown      empty        unconfigured ok
usb3/6                         unknown      empty        unconfigured ok
usb4/1                         unknown      empty        unconfigured ok
usb4/2                         unknown      empty        unconfigured ok
usb5/1                         unknown      empty        unconfigured ok
usb5/2                         unknown      empty        unconfigured ok
usb6/1                         unknown      empty        unconfigured ok
usb6/2                         unknown      empty        unconfigured ok
usb7/1                         unknown      empty        unconfigured ok
usb7/2                         unknown      empty        unconfigured ok
usb7/3                         unknown      empty        unconfigured ok
usb7/4                         unknown      empty        unconfigured ok
usb7/5                         unknown      empty        unconfigured ok
usb7/6                         unknown      empty        unconfigured ok
HBWHCG01:/# cfgadm -c configure c1
HBWHCG01:/# cfgadm -c configure c2
HBWHCG01:/#
HBWHCG01:/# powermt config
HBWHCG01:/# powercf config
HBWHCG01:/# powercf -q
HBWHCG01:/# powermt save
HBWHCG01:/# /etc/powermt display dev=emcpower0a
Pseudo name=emcpower0a
CLARiiON ID=FCNMM124800004 [MMSUN]
Logical device ID=6006016068EC3200C4B6FD8ADB9BE211
state=dead; policy=CLAROpt; priority=0; queued-IOs=0;
Owner: default=Unknown, current=Unknown Array failover mode: 1
==============================================================================
--------------- Host ---------------   - Stor -   -- I/O Path --  -- Stats ---
###  HW Path               I/O Paths    Interf.   Mode    State   Q-IOs Errors
==============================================================================
3072 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0 c1t0d0s0  SP A1     active  dead        0      0
3073 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0 c2t0d0s0  SP B1     active  dead        0      0



HBWHCG01:/# prtvtoc /dev/rdsk/emcpower0a
prtvtoc: /dev/rdsk/emcpower0a: Unable to read Disk geometry errno = 0x5
HBWHCG01:/# ls -la /dev/*dsk/emcpower0*
lrwxrwxrwx   1 root     root          34 Apr  4 22:16 /dev/dsk/emcpower0 -> ../../devices/pseudo/emcp@0:wd,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0a -> ../../devices/pseudo/emcp@0:a,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0b -> ../../devices/pseudo/emcp@0:b,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0c -> ../../devices/pseudo/emcp@0:c,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0d -> ../../devices/pseudo/emcp@0:d,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0e -> ../../devices/pseudo/emcp@0:e,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0f -> ../../devices/pseudo/emcp@0:f,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0g -> ../../devices/pseudo/emcp@0:g,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0i -> ../../devices/pseudo/emcp@0:i,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0j -> ../../devices/pseudo/emcp@0:j,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0k -> ../../devices/pseudo/emcp@0:k,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0l -> ../../devices/pseudo/emcp@0:l,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0m -> ../../devices/pseudo/emcp@0:m,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0n -> ../../devices/pseudo/emcp@0:n,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0o -> ../../devices/pseudo/emcp@0:o,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0p -> ../../devices/pseudo/emcp@0:p,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0p0 -> ../../devices/pseudo/emcp@0:q,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0p1 -> ../../devices/pseudo/emcp@0:r,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0p2 -> ../../devices/pseudo/emcp@0:s,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0p3 -> ../../devices/pseudo/emcp@0:t,blk
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/dsk/emcpower0p4 -> ../../devices/pseudo/emcp@0:u,blk
lrwxrwxrwx   1 root     root          34 Apr  4 22:16 /dev/rdsk/emcpower0 -> ../../devices/pseudo/emcp@0:wd,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0a -> ../../devices/pseudo/emcp@0:a,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0b -> ../../devices/pseudo/emcp@0:b,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0c -> ../../devices/pseudo/emcp@0:c,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0d -> ../../devices/pseudo/emcp@0:d,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0e -> ../../devices/pseudo/emcp@0:e,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0f -> ../../devices/pseudo/emcp@0:f,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0g -> ../../devices/pseudo/emcp@0:g,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0i -> ../../devices/pseudo/emcp@0:i,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0j -> ../../devices/pseudo/emcp@0:j,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0k -> ../../devices/pseudo/emcp@0:k,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0l -> ../../devices/pseudo/emcp@0:l,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0m -> ../../devices/pseudo/emcp@0:m,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0n -> ../../devices/pseudo/emcp@0:n,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0o -> ../../devices/pseudo/emcp@0:o,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0p -> ../../devices/pseudo/emcp@0:p,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0p0 -> ../../devices/pseudo/emcp@0:q,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0p1 -> ../../devices/pseudo/emcp@0:r,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0p2 -> ../../devices/pseudo/emcp@0:s,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0p3 -> ../../devices/pseudo/emcp@0:t,raw
lrwxrwxrwx   1 root     root          33 Apr  4 14:51 /dev/rdsk/emcpower0p4 -> ../../devices/pseudo/emcp@0:u,raw
HBWHCG01:/# vxdisk scandisks
HBWHCG01:/# vxdisk list emcpower0s2
VxVM vxdisk ERROR V-5-1-558 Disk emcpower0s2: Disk not in the configuration

g_lee
Level 6

maxpaynebupt:

The bigger issue appears to be that all paths to the disk are showing dead in powermt display - so the OS & powerpath cannot see this device properly, so VxVM can't see it until that is fixed.

HBWHCG01:/# /etc/powermt display dev=emcpower0a
Pseudo name=emcpower0a
CLARiiON ID=FCNMM124800004 [MMSUN]
Logical device ID=6006016068EC3200C4B6FD8ADB9BE211
state=dead; policy=CLAROpt; priority=0; queued-IOs=0;
Owner: default=Unknown, current=Unknown Array failover mode: 1
==============================================================================
--------------- Host ---------------   - Stor -   -- I/O Path --  -- Stats ---
###  HW Path               I/O Paths    Interf.   Mode    State   Q-IOs Errors
==============================================================================
3072 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0 c1t0d0s0  SP A1     active  dead        0      0
3073 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0 c2t0d0s0  SP B1     active  dead        0      0

Is emcpower0 definitely the correct disk? There are no devices in the latest format output for emcpower0 or c[12]t0d0 - so the OS isn't seeing the disk either.

Have you tried cleaning the OS device tree?

# devfsadm -Csv   #### -s doesn't perform the clean, but you can see if will do anything

If you're sure this is the correct disk:

# devfsadm -Cv ### clean up

# /etc/powermt check  ### if devices are dead it will prompt you to remove them

Then check the array / server to ensure the disks show up correctly in format, and then try to rescan OS -> powerpath -> vxdisk again

 

EDIT: Additional check to see if using persistent naming:

# vxddladm get namingscheme

Note this still won't fix/affect anything until the OS & powerpath see the disk properly, so you still need to fix that first before attempting to rescan VxVM

maxpaynebupt
Level 3

thanks, 

I used the" devfsadm -Cv" and "/etc/powermt check ” to clean up and then used

powermt config

powermt save

powercf -q

 to get the pseudo device back.

 

And I came back to the error I came to at the very beginning:

 

 

 

HBWHCG01:/opt/enh_install_LS217/lib/SunOS# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
emcpower0s2  auto            -            -            error
emcpower1s2  auto:none       -            -            online invalid
emcpower2s2  auto:none       -            -            online invalid
emcpower3s2  auto:none       -            -            online invalid
emcpower4s2  auto:none       -            -            online invalid
emcpower5s2  auto:none       -            -            online invalid
emcpower6s2  auto:none       -            -            online invalid

HBWHCG01:/opt/enh_install_LS217/lib/SunOS# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <DEFAULT cyl 36348 alt 2 hd 255 sec 126>
          /pci@0,0/pci8086,3408@1/pci1000,9262@0/sd@0,0
       1. c1t0d0 <DGC-RAID 10-0223-2.32TB>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,0
       2. c1t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,1
       3. c1t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,2
       4. c1t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,3
       5. c1t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,4
       6. c1t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,5
       7. c1t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,6
       8. c2t0d0 <DGC-RAID 10-0223-2.32TB>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,0
       9. c2t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,1
      10. c2t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,2
      11. c2t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,3
      12. c2t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,4
      13. c2t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,5
      14. c2t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,6
      15. emcpower0a <DGC-RAID 10-0223-2.32TB>
          /pseudo/emcp@0
      16. emcpower1a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@1
      17. emcpower2a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@2
      18. emcpower3a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@3
      19. emcpower4a <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
          /pseudo/emcp@4
      20. emcpower5a <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
          /pseudo/emcp@5
      21. emcpower6a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
          /pseudo/emcp@6
Specify disk (enter its number): ^C
HBWHCG01:/opt/enh_install_LS217/lib/SunOS#

HBWHCG01:/opt/enh_install_LS217/lib/SunOS# format -e /dev/vx/rdmp/emcpower0s2
No disks found!

HBWHCG01:initdisk=/etc/vx/bin/vxdisksetup -if emcpower0 format=sliced
VxVM vxdisksetup ERROR V-5-2-43 emcpower0: Invalid disk device for vxdisksetup

maxpaynebupt
Level 3

Solved.

 

It's all because of our Software's install script, it was trying to use fdisk -B to limit the size of the disk to 2TB, and after the fdisk attempt, the disk become error status.............

I tried to comment out the fdisk -B line, after this, the vxdisksetup and createdg works fine!

 

 

Thank u  everyone who helped me , thanks.

g_lee
Level 6

Glad the problem has been resolved.

fyi - in your output it looks like you've run

HBWHCG01:/opt/enh_install_LS217/lib/SunOS# format -e /dev/vx/rdmp/emcpower0s2
No disks found!

For low-level format/fdisk/labelling commands it's generally better/recommended to run on the underlying OS device (ie: c1t0d0) rather than pseudo devices (empower0 or /dev/vx/rdmp/emcpower0s2) - particularly on x86. ie: label the underlying OS device, then query (eg: prtvtoc) the pseudo device and it should pick up the change - then you can go on to use the pseudo device.