β04-03-2013 09:18 PM
Hi,
I have tried to bring up this lun for 2 days, and got nearly no progress, sometimes the lun can be online, but if I try to use vxdisksetup to setup it, it would become error again, I am really despairing, I hope someone can help me to solve this, thank u.
HBWHCG01:/# vxdisk list
DEVICE TYPE DISK GROUP STATUS
c0t0d0s2 auto:none - - online invalid
emcpower0s2 auto - - error
emcpower1s2 auto:none - - online invalid
emcpower2s2 auto:none - - online invalid
emcpower3s2 auto:none - - online invalid
emcpower4s2 auto:none - - online invalid
emcpower5s2 auto:none - - online invalid
emcpower6s2 auto:none - - online invalid
HBWHCG01:/# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <DEFAULT cyl 36348 alt 2 hd 255 sec 126>
/pci@0,0/pci8086,3408@1/pci1000,9262@0/sd@0,0
1. c1t0d0 <DGC-RAID 10-0223-2.32TB>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,0
2. c1t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,1
3. c1t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,2
4. c1t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,3
5. c1t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,4
6. c1t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,5
7. c1t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,6
8. c2t0d0 <DGC-RAID 10-0223-2.32TB>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,0
9. c2t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,1
10. c2t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,2
11. c2t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,3
12. c2t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,4
13. c2t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,5
14. c2t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,6
15. emcpower0a <DGC-RAID 10-0223-2.32TB>
/pseudo/emcp@0
16. emcpower1a <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pseudo/emcp@1
17. emcpower2a <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pseudo/emcp@2
18. emcpower3a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@3
19. emcpower4a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@4
20. emcpower5a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@5
21. emcpower6a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@6
Specify disk (enter its number): 15
selecting emcpower0a
[disk formatted]
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
fdisk - run the fdisk program
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
inquiry - show vendor, product and revision
volname - set 8-character volume name
!<cmd> - execute <cmd>, then return
quit
format> p
PARTITION MENU:
0 - change `0' partition
1 - change `1' partition
2 - change `2' partition
3 - change `3' partition
4 - change `4' partition
5 - change `5' partition
6 - change `6' partition
expand - expand label to use whole disk
select - select a predefined table
modify - modify a predefined partition table
name - name the current table
print - display the current table
label - write partition map and label to the disk
!<cmd> - execute <cmd>, then return
quit
partition> p
Current partition table (original):
Total disk sectors available: 4991205341 + 16384 (reserved sectors)
Part Tag Flag First Sector Size Last Sector
0 usr wm 34 2.32TB 4991205341
1 unassigned wm 0 0 0
2 unassigned wm 0 0 0
3 unassigned wm 0 0 0
4 unassigned wm 0 0 0
5 unassigned wm 0 0 0
6 unassigned wm 0 0 0
8 reserved wm 4991205342 8.00MB 4991221725
Solved! Go to Solution.
β04-04-2013 07:41 AM
maxpaynebupt:
The bigger issue appears to be that all paths to the disk are showing dead in powermt display - so the OS & powerpath cannot see this device properly, so VxVM can't see it until that is fixed.
HBWHCG01:/# /etc/powermt display dev=emcpower0a
Pseudo name=emcpower0a
CLARiiON ID=FCNMM124800004 [MMSUN]
Logical device ID=6006016068EC3200C4B6FD8ADB9BE211
state=dead; policy=CLAROpt; priority=0; queued-IOs=0;
Owner: default=Unknown, current=Unknown Array failover mode: 1
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0 c1t0d0s0 SP A1 active dead 0 0
3073 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0 c2t0d0s0 SP B1 active dead 0 0
Is emcpower0 definitely the correct disk? There are no devices in the latest format output for emcpower0 or c[12]t0d0 - so the OS isn't seeing the disk either.
Have you tried cleaning the OS device tree?
# devfsadm -Csv #### -s doesn't perform the clean, but you can see if will do anything
If you're sure this is the correct disk:
# devfsadm -Cv ### clean up
# /etc/powermt check ### if devices are dead it will prompt you to remove them
Then check the array / server to ensure the disks show up correctly in format, and then try to rescan OS -> powerpath -> vxdisk again
EDIT: Additional check to see if using persistent naming:
# vxddladm get namingscheme
Note this still won't fix/affect anything until the OS & powerpath see the disk properly, so you still need to fix that first before attempting to rescan VxVM
β04-03-2013 09:20 PM
And , it's a standalone server with a EMC ax4-5 disk array connected by two fc fibre.
β04-03-2013 10:31 PM
You did not mention your SF version?
Try this procedure:
First, format the lun using the /dev/vx/rdmp name, and set the size of partition 0 to 0tb, and label the disk.
# format -e /dev/vx/rdmp/emcpower#
Run vxdisksetup against the disk.
# /etc/vx/bin/vxdisksetup -i -f emcpower# format=sliced
Run vxdisk init.
# vxdisk -f init emcpower# format=sliced
Then create a dg using that disk as the first disk.
# vxdg init test_dg cds=off testd1=emcpower#
β04-04-2013 12:05 AM
thanks for your reply^_^
Is a 2.3t lun too big for vx? Do I need to resize it?
I already tried to resize the lun to 125GB, but stuck at the system can not recognize the new lun........
Ok, I will try to resize the lun back to 2380GB, and try it again, but the rebound of the emc would take about 6 hours, I will come back after that, thank u.
β04-04-2013 12:45 AM
β04-04-2013 12:50 AM
It seems the vxVM is still trying to find the old lun:
β04-04-2013 12:51 AM
HBWHCG01:/# pkginfo -l vxvm
ERROR: information for "vxvm" was not found
HBWHCG01:/# pkginfo -l VRTSvxvm
PKGINST: VRTSvxvm
NAME: Binaries for VERITAS Volume Manager by Symantec
CATEGORY: system
ARCH: i386
VERSION: 5.1,REV=10.06.2009.21.48
BASEDIR: /
VENDOR: Symantec Corporation
DESC: Virtual Disk Subsystem
PSTAMP: 5.1.102.000-5.1SP1RP2_x86-2011-09-13
INSTDATE: Apr 02 2012 14:51
HOTLINE: http://support.veritas.com/phonesup/phonesup_ddProduct_.htm
EMAIL: support@veritas.com
STATUS: completely installed
FILES: 875 installed pathnames
44 shared pathnames
112 directories
402 executables
375553 blocks used (approx)
HBWHCG01:/#
β04-04-2013 05:35 AM
What version of Powerpath are you using?
# pkginfo -l EMCpower
Try the following to remove/rescan to pick up the disk:
# ps -ef |grep vxesd
#### if vxesd is running, stop it while you clean up the config
# vxddladm stop eventsource
#### now remove the disk:
# vxdisk rm emcpower0s2
# vxdisk list ### check emcpower0 is no longer listed
#### after you resize the lun, rescan OS to pick up
# cfgadm -al
# cfgadm -c configure c#
### rescan powerpath
# /etc/powermt config
# /etc/powercf config
# /etc/powercf -q
# /etc/powermt save
#### check the following outputs:
# /etc/powermt display dev=emcpower0a
# prtvtoc /dev/rdsk/emcpower0a
# ls -la /dev/*dsk/emcpower0* ### check powerdevices created correctly as this can be issue in old versions of powerpath
#### If prtvtoc shows the correct size:
# vxdisk scandisks
# vxdisk list emcpower0s2
β04-04-2013 07:27 AM
Hi,
It seems to not work, there should be some database of vxdisk store the lun info and prevent the system from regain the expanded new lun.
HBWHCG01:/# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <DEFAULT cyl 36348 alt 2 hd 255 sec 126>
/pci@0,0/pci8086,3408@1/pci1000,9262@0/sd@0,0
1. c1t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,1
2. c1t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,2
3. c1t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,3
4. c1t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,4
5. c1t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,5
6. c1t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,6
7. c2t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,1
8. c2t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,2
9. c2t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,3
10. c2t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,4
11. c2t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,5
12. c2t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,6
13. emcpower1a <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pseudo/emcp@1
14. emcpower2a <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pseudo/emcp@2
15. emcpower3a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@3
16. emcpower4a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@4
17. emcpower5a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@5
18. emcpower6a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@6
Specify disk (enter its number): ^C
HBWHCG01:/#
HBWHCG01:/#
HBWHCG01:/# pkginfo -l EMCpower
PKGINST: EMCpower
NAME: EMC PowerPath
CATEGORY: system
ARCH: i386
VERSION: 5.3.P03_b003
BASEDIR: /opt
VENDOR: EMC Corporation
DESC: EMC Powerpath Multipathing, version 5.3.P03_b003
PSTAMP: dev:2011-12-27T07:23:25EST
INSTDATE: Apr 04 2013 04:24
STATUS: completely installed
FILES: 148 installed pathnames
34 directories
67 executables
68203 blocks used (approx)
HBWHCG01:/# ps -ef |grep vxesd
root 814 1 0 22:16:24 ? 0:00 /sbin/vxesd -x dmp_monitor_osevent=off
root 4243 4104 0 22:17:35 pts/2 0:00 grep vxesd
HBWHCG01:/#
HBWHCG01:/# vxddladm stop eventsource
HBWHCG01:/# vxdisk rm emcpower0s2
VxVM vxdisk ERROR V-5-1-639 Failed to obtain locks:
emcpower0s2: no such object in the configuration
HBWHCG01:/# vxdisk list
DEVICE TYPE DISK GROUP STATUS
c0t0d0s2 auto:none - - online invalid
emcpower1s2 auto:none - - online invalid
emcpower2s2 auto:none - - online invalid
emcpower3s2 auto:none - - online invalid
emcpower4s2 auto:none - - online invalid
emcpower5s2 auto:none - - online invalid
emcpower6s2 auto:none - - online invalid
HBWHCG01:/# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
Slot1 pci-pci/hp connected configured ok
Slot2 pci-pci/hp connected configured ok
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c1 fc-private connected configured unknown
c1::500601613ba07373 disk connected configured unknown
c2 fc-private connected configured unknown
c2::500601693ba07373 disk connected configured unknown
pcie3 unknown empty unconfigured unknown
pcie4 unknown empty unconfigured unknown
pcieb5.pcie0 etherne/hp connected configured ok
sata0/0 sata-port empty unconfigured ok
sata0/1 sata-port empty unconfigured ok
sata0/2 sata-port empty unconfigured ok
sata0/3 sata-port empty unconfigured ok
sata0/4 sata-port empty unconfigured ok
sata0/5 sata-port empty unconfigured ok
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
usb1/1 unknown empty unconfigured ok
usb1/2 unknown empty unconfigured ok
usb2/1 usb-device connected configured ok
usb2/2 unknown empty unconfigured ok
usb3/1 unknown empty unconfigured ok
usb3/2 unknown empty unconfigured ok
usb3/3 usb-hub connected configured ok
usb3/3.1 unknown empty unconfigured ok
usb3/3.2 unknown empty unconfigured ok
usb3/3.3 unknown empty unconfigured ok
usb3/4 unknown empty unconfigured ok
usb3/5 unknown empty unconfigured ok
usb3/6 unknown empty unconfigured ok
usb4/1 unknown empty unconfigured ok
usb4/2 unknown empty unconfigured ok
usb5/1 unknown empty unconfigured ok
usb5/2 unknown empty unconfigured ok
usb6/1 unknown empty unconfigured ok
usb6/2 unknown empty unconfigured ok
usb7/1 unknown empty unconfigured ok
usb7/2 unknown empty unconfigured ok
usb7/3 unknown empty unconfigured ok
usb7/4 unknown empty unconfigured ok
usb7/5 unknown empty unconfigured ok
usb7/6 unknown empty unconfigured ok
HBWHCG01:/# cfgadm -c configure c1
HBWHCG01:/# cfgadm -c configure c2
HBWHCG01:/#
HBWHCG01:/# powermt config
HBWHCG01:/# powercf config
HBWHCG01:/# powercf -q
HBWHCG01:/# powermt save
HBWHCG01:/# /etc/powermt display dev=emcpower0a
Pseudo name=emcpower0a
CLARiiON ID=FCNMM124800004 [MMSUN]
Logical device ID=6006016068EC3200C4B6FD8ADB9BE211
state=dead; policy=CLAROpt; priority=0; queued-IOs=0;
Owner: default=Unknown, current=Unknown Array failover mode: 1
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0 c1t0d0s0 SP A1 active dead 0 0
3073 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0 c2t0d0s0 SP B1 active dead 0 0
HBWHCG01:/# prtvtoc /dev/rdsk/emcpower0a
prtvtoc: /dev/rdsk/emcpower0a: Unable to read Disk geometry errno = 0x5
HBWHCG01:/# ls -la /dev/*dsk/emcpower0*
lrwxrwxrwx 1 root root 34 Apr 4 22:16 /dev/dsk/emcpower0 -> ../../devices/pseudo/emcp@0:wd,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0a -> ../../devices/pseudo/emcp@0:a,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0b -> ../../devices/pseudo/emcp@0:b,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0c -> ../../devices/pseudo/emcp@0:c,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0d -> ../../devices/pseudo/emcp@0:d,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0e -> ../../devices/pseudo/emcp@0:e,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0f -> ../../devices/pseudo/emcp@0:f,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0g -> ../../devices/pseudo/emcp@0:g,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0i -> ../../devices/pseudo/emcp@0:i,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0j -> ../../devices/pseudo/emcp@0:j,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0k -> ../../devices/pseudo/emcp@0:k,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0l -> ../../devices/pseudo/emcp@0:l,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0m -> ../../devices/pseudo/emcp@0:m,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0n -> ../../devices/pseudo/emcp@0:n,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0o -> ../../devices/pseudo/emcp@0:o,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0p -> ../../devices/pseudo/emcp@0:p,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0p0 -> ../../devices/pseudo/emcp@0:q,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0p1 -> ../../devices/pseudo/emcp@0:r,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0p2 -> ../../devices/pseudo/emcp@0:s,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0p3 -> ../../devices/pseudo/emcp@0:t,blk
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/dsk/emcpower0p4 -> ../../devices/pseudo/emcp@0:u,blk
lrwxrwxrwx 1 root root 34 Apr 4 22:16 /dev/rdsk/emcpower0 -> ../../devices/pseudo/emcp@0:wd,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0a -> ../../devices/pseudo/emcp@0:a,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0b -> ../../devices/pseudo/emcp@0:b,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0c -> ../../devices/pseudo/emcp@0:c,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0d -> ../../devices/pseudo/emcp@0:d,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0e -> ../../devices/pseudo/emcp@0:e,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0f -> ../../devices/pseudo/emcp@0:f,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0g -> ../../devices/pseudo/emcp@0:g,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0i -> ../../devices/pseudo/emcp@0:i,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0j -> ../../devices/pseudo/emcp@0:j,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0k -> ../../devices/pseudo/emcp@0:k,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0l -> ../../devices/pseudo/emcp@0:l,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0m -> ../../devices/pseudo/emcp@0:m,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0n -> ../../devices/pseudo/emcp@0:n,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0o -> ../../devices/pseudo/emcp@0:o,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0p -> ../../devices/pseudo/emcp@0:p,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0p0 -> ../../devices/pseudo/emcp@0:q,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0p1 -> ../../devices/pseudo/emcp@0:r,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0p2 -> ../../devices/pseudo/emcp@0:s,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0p3 -> ../../devices/pseudo/emcp@0:t,raw
lrwxrwxrwx 1 root root 33 Apr 4 14:51 /dev/rdsk/emcpower0p4 -> ../../devices/pseudo/emcp@0:u,raw
HBWHCG01:/# vxdisk scandisks
HBWHCG01:/# vxdisk list emcpower0s2
VxVM vxdisk ERROR V-5-1-558 Disk emcpower0s2: Disk not in the configuration
β04-04-2013 07:41 AM
maxpaynebupt:
The bigger issue appears to be that all paths to the disk are showing dead in powermt display - so the OS & powerpath cannot see this device properly, so VxVM can't see it until that is fixed.
HBWHCG01:/# /etc/powermt display dev=emcpower0a
Pseudo name=emcpower0a
CLARiiON ID=FCNMM124800004 [MMSUN]
Logical device ID=6006016068EC3200C4B6FD8ADB9BE211
state=dead; policy=CLAROpt; priority=0; queued-IOs=0;
Owner: default=Unknown, current=Unknown Array failover mode: 1
==============================================================================
--------------- Host --------------- - Stor - -- I/O Path -- -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors
==============================================================================
3072 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0 c1t0d0s0 SP A1 active dead 0 0
3073 pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0 c2t0d0s0 SP B1 active dead 0 0
Is emcpower0 definitely the correct disk? There are no devices in the latest format output for emcpower0 or c[12]t0d0 - so the OS isn't seeing the disk either.
Have you tried cleaning the OS device tree?
# devfsadm -Csv #### -s doesn't perform the clean, but you can see if will do anything
If you're sure this is the correct disk:
# devfsadm -Cv ### clean up
# /etc/powermt check ### if devices are dead it will prompt you to remove them
Then check the array / server to ensure the disks show up correctly in format, and then try to rescan OS -> powerpath -> vxdisk again
EDIT: Additional check to see if using persistent naming:
# vxddladm get namingscheme
Note this still won't fix/affect anything until the OS & powerpath see the disk properly, so you still need to fix that first before attempting to rescan VxVM
β04-04-2013 08:58 AM
thanks,
I used the" devfsadm -Cv" and "/etc/powermt check β to clean up and then used
powermt config
powermt save
powercf -q
to get the pseudo device back.
And I came back to the error I came to at the very beginningοΌ
HBWHCG01:/opt/enh_install_LS217/lib/SunOS# vxdisk list
DEVICE TYPE DISK GROUP STATUS
c0t0d0s2 auto:none - - online invalid
emcpower0s2 auto - - error
emcpower1s2 auto:none - - online invalid
emcpower2s2 auto:none - - online invalid
emcpower3s2 auto:none - - online invalid
emcpower4s2 auto:none - - online invalid
emcpower5s2 auto:none - - online invalid
emcpower6s2 auto:none - - online invalid
HBWHCG01:/opt/enh_install_LS217/lib/SunOS# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <DEFAULT cyl 36348 alt 2 hd 255 sec 126>
/pci@0,0/pci8086,3408@1/pci1000,9262@0/sd@0,0
1. c1t0d0 <DGC-RAID 10-0223-2.32TB>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,0
2. c1t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,1
3. c1t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,2
4. c1t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,3
5. c1t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,4
6. c1t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,5
7. c1t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0/fp@0,0/disk@w500601613ba07373,6
8. c2t0d0 <DGC-RAID 10-0223-2.32TB>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,0
9. c2t0d1 <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,1
10. c2t0d2 <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,2
11. c2t0d3 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,3
12. c2t0d4 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,4
13. c2t0d5 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,5
14. c2t0d6 <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pci@0,0/pci8086,340c@5/pci111d,806c@0/pci111d,806c@2/pci1077,172@0,1/fp@0,0/disk@w500601693ba07373,6
15. emcpower0a <DGC-RAID 10-0223-2.32TB>
/pseudo/emcp@0
16. emcpower1a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@1
17. emcpower2a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@2
18. emcpower3a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@3
19. emcpower4a <DEFAULT cyl 6524 alt 2 hd 255 sec 63>
/pseudo/emcp@4
20. emcpower5a <DEFAULT cyl 3913 alt 2 hd 255 sec 63>
/pseudo/emcp@5
21. emcpower6a <DEFAULT cyl 1020 alt 2 hd 64 sec 32>
/pseudo/emcp@6
Specify disk (enter its number): ^C
HBWHCG01:/opt/enh_install_LS217/lib/SunOS#
HBWHCG01:/opt/enh_install_LS217/lib/SunOS# format -e /dev/vx/rdmp/emcpower0s2
No disks found!
HBWHCG01:initdisk=/etc/vx/bin/vxdisksetup -if emcpower0 format=sliced
VxVM vxdisksetup ERROR V-5-2-43 emcpower0: Invalid disk device for vxdisksetup
β04-04-2013 07:25 PM
Solved.
It's all because of our Software's install script, it was trying to use fdisk -B to limit the size of the disk to 2TB, and after the fdisk attempt, the disk become error status.............
I tried to comment out the fdisk -B line, after this, the vxdisksetup and createdg works fine!
Thank u everyone who helped me , thanks.
β04-05-2013 04:02 AM
Glad the problem has been resolved.
fyi - in your output it looks like you've run
HBWHCG01:/opt/enh_install_LS217/lib/SunOS# format -e /dev/vx/rdmp/emcpower0s2 No disks found!
For low-level format/fdisk/labelling commands it's generally better/recommended to run on the underlying OS device (ie: c1t0d0) rather than pseudo devices (empower0 or /dev/vx/rdmp/emcpower0s2) - particularly on x86. ie: label the underlying OS device, then query (eg: prtvtoc) the pseudo device and it should pick up the change - then you can go on to use the pseudo device.