Vxdiskadm Replace disk does not show my failed dis...

chuckchang23 · ‎05-08-2013

Hello all,

i would like to replace my failed disk via the vxdiskadm utility

EMMDPD04:/# vxdisk list

DEVICE TYPE DISK GROUP STATUS

c0t0d0s2 auto:sliced rootdisk rootdg online

c0t1d0s2 auto:sliced rootmirror rootdg online

c0t2d0s2 auto - - error

c0t3d0s2 auto:none - - online invalid

emcpower0s2 auto:sliced bgw1dgemcpower0 bgw1dg online shared

emcpower1s2 auto:sliced ora1dgemcpower7 ora1dg online shared

emcpower2s2 auto:sliced fmm1dgemcpower6 fmm1dg online shared

emcpower3s2 auto:sliced bgw1dgemcpower1 bgw1dg online shared

emcpower4s2 auto:cdsdisk - - online

emcpower5s2 auto:cdsdisk - - online

emcpower6s2 auto:sliced lic1dgemcpower5 lic1dg online shared

emcpower7s2 auto:cdsdisk - - online

emcpower8s2 auto:sliced bgw1dgemcpower8 bgw1dg online shared

emcpower9s2 auto:sliced bgw1dgemcpower9 bgw1dg online shared

emcpower10s2 auto:sliced bgw1dgemcpower10 bgw1dg online shared

emcpower11s2 auto:sliced bgw1dgemcpower11 bgw1dg online shared

emcpower12s2 auto:sliced bgw1dgemcpower12 bgw1dg online shared

emcpower13s2 auto:sliced bgw1dgemcpower13 bgw1dg online shared

emcpower14s2 auto:sliced bgw1dgemcpower14 bgw1dg online shared

emcpower15s2 auto:sliced bgw1dgemcpower15 bgw1dg online shared

EMMDPD04:/#

EMMDPD04:/# vxdiskadm

Volume Manager Support Operations

Menu: VolumeManager/Disk

1 Add or initialize one or more disks

2 Encapsulate one or more disks

3 Remove a disk

4 Remove a disk for replacement

5 Replace a failed or removed disk

6 Mirror volumes on a disk

7 Move volumes from a disk

8 Enable access to (import) a disk group

9 Remove access to (deport) a disk group

10 Enable (online) a disk device

11 Disable (offline) a disk device

12 Mark a disk as a spare for a disk group

13 Turn off the spare flag on a disk

14 Unrelocate subdisks back to a disk

15 Exclude a disk from hot-relocation use

16 Make a disk available for hot-relocation use

17 Prevent multipathing/Suppress devices from VxVM's view

18 Allow multipathing/Unsuppress devices from VxVM's view

19 List currently suppressed/non-multipathed devices

20 Change the disk naming scheme

21 Get the newly connected/zoned disks in VxVM view

22 Change/Display the default disk layouts

23 Mark a disk as allocator-reserved for a disk group

24 Turn off the allocator-reserved flag on a disk

list List disk information

? Display help about menu

?? Display help about the menuing system

q Exit from menus

Select an operation to perform: 4

Remove a disk for replacement

Menu: VolumeManager/Disk/RemoveForReplace

Use this menu operation to remove a physical disk from a disk

group, while retaining the disk name. This changes the state

for the disk name to a "removed" disk. If there are any

initialized disks that are not part of a disk group, you will be

given the option of using one of these disks as a replacement.

Enter disk name [<disk>,list,q,?] list

Disk group: rootdg

DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE

dm rootdisk c0t0d0s2 auto 80321 583834230 -

dm rootmirror c0t1d0s2 auto 80321 583850295 -

Disk group: bgw1dg

DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE

dm bgw1dgemcpower0 emcpower0s2 auto 96130 1312574752 -

dm bgw1dgemcpower1 emcpower3s2 auto 96130 1310486288 -

dm bgw1dgemcpower8 emcpower8s2 auto 96130 1576844000 -

dm bgw1dgemcpower9 emcpower9s2 auto 96130 1576844000 -

dm bgw1dgemcpower10 emcpower10s2 auto 96130 1576844000 -

dm bgw1dgemcpower11 emcpower11s2 auto 96130 1576844000 -

dm bgw1dgemcpower12 emcpower12s2 auto 96130 1314695328 -

dm bgw1dgemcpower13 emcpower13s2 auto 96130 1293746560 -

dm bgw1dgemcpower14 emcpower14s2 auto 96130 1576844000 -

dm bgw1dgemcpower15 emcpower15s2 auto 96130 1576844000 -

Disk group: fmm1dg

DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE

dm fmm1dgemcpower6 emcpower2s2 auto 80065 104711648 -

Disk group: lic1dg

DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE

dm lic1dgemcpower5 emcpower6s2 auto 67324 2019328 -

Disk group: ora1dg

DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE

dm ora1dgemcpower7 emcpower1s2 auto 80065 419280416 -

Enter disk name [<disk>,list,q,?] c0t2d0s2

VxVM ERROR V-5-2-400

There is no disk named c0t2d0s2 in any disk group configuration.

To get a list of disks enter "list".

Enter disk name [<disk>,list,q,?] q

Is this the proper way or is there any other way to replace my spare disk?

starflyfly · ‎05-08-2013

Hi,

In vxvm admin guide document, there is a chapter for replace disks, Except upper operation, you need run some os commands to clean device tree.

You can access sort.symantec.com to check document.

https://sort.symantec.com/documents

Regards

starflyfly · ‎05-08-2013

Like solaris , sf5.1sp1:

https://sort.symantec.com/public/documents/sfha/5.1sp1/solaris/productguides/html/vxvm_admin/ch05.ht...

chuckchang23 · ‎05-08-2013

Hi starflyfly,

when i use the vxdiskadm command, is says it needs a diskgroup the problem is that the spare disk has no diskgroup

stinsong · ‎05-08-2013

Hi Chunckchang23,

Did you try menu option 5 VolumeManager/Disk/ReplaceDisk? It is used for your condition.

Menu: VolumeManager/Disk/ReplaceDisk

Use this menu operation to specify a replacement disk for a disk

that you removed with the "Remove a disk for replacement" menu

operation, or that failed during use. You will be prompted for

a disk name to replace and a disk device to use as a replacement.

You can choose an uninitialized disk, in which case the disk will

be initialized, or you can choose a disk that you have already

initialized using the Add or initialize a disk menu operation.

chuckchang23 · ‎05-09-2013

Hi Starflyfly

I cannot proceed with this step due to the error can't be detected.

Pls see output of vxdiskadm

Replace a failed or removed disk

Menu: VolumeManager/Disk/ReplaceDisk

Use this menu operation to specify a replacement disk for a disk

that you removed with the "Remove a disk for replacement" menu

operation, or that failed during use. You will be prompted for

a disk name to replace and a disk device to use as a replacement.

You can choose an uninitialized disk, in which case the disk will

be initialized, or you can choose a disk that you have already

initialized using the Add or initialize a disk menu operation.

VxVM INFO V-5-2-396

There are currently no devices that are failed or removed.

Hit RETURN to continue.

kjbss · ‎05-09-2013

Make sure your OS can see the c0t2d0s2 disk BEFORE you try any volume manager actions.

Does 'prtvtoc /dev/rdsk/c0t2d0s2' show you the contents of the label, or an error?

If an error is shown, you need to fix your OS to be able to see your disk; try 'devfsadm -Ccv disk', then try the prtvtoc command again, and depending on if that work or not, you might try to format the disk, then make sure you can prtvtoc it.

If that doesn't work, you may need to physically replace the drive (or is that what you have already done)?

You sould also keep on eye on the OS console log (/var/adm/messages) and address any errors reported there.

Only after you have fixed and/or proven access from the OS level should you start using VxVM tools on the device.

Once your OS tools can successfullly see the disk, you should do 'vxdisk scandisks' (or 'vxdctl enable') so that VxVM can refresh his device tables with the new and/or fixed devices.

chuckchang23 · ‎05-09-2013

Hi kjbss

It can be seen with the prtvoc command.

EMMDPD03:/# prtvtoc /dev/rdsk/c0t2d0s2

* /dev/rdsk/c0t2d0s2 partition map

*

* Dimensions:

* 512 bytes/sector

* 63 sectors/track

* 255 tracks/cylinder

* 16065 sectors/cylinder

* 36350 cylinders

* 36348 accessible cylinders

*

* Flags:

* 1: unmountable

* 10: read-only

*

* Unallocated space:

* First Sector Last

* Sector Count Sector

* 0 16065 16064

*

* First Sector Last

* Partition Tag Flags Sector Count Sector Mount Directory

3 9 00 16065 10490445 10506509 /opt/bmc

6 11 00 10506510 573424110 583930619 /backup

8 1 01 0 16065 16064

EMMDPD03:/#

my problem is that when i list down all disks, it shows the disk with an error state,

EMMDPD04:/# vxdisk list

DEVICE TYPE DISK GROUP STATUS

c0t0d0s2 auto:sliced rootdisk rootdg online

c0t1d0s2 auto:sliced rootmirror rootdg online

c0t2d0s2 auto - - error <---------------------- Error disk

c0t3d0s2 auto:none - - online invalid

but when i try to fix this, it can't be shown on the VVM. Hope you could help me clear the error

starflyfly · ‎05-09-2013

Hi,

The disk is in error state, so you can't use replace failed disk procedures to replace it.

the error state is because you have no partition 2 in this disk.

If you dont' need keep original data on this disk,

use format to create partition 2 for this disk, which from 0 to last sector.

then , run:

vxdctl enable,

it should in online invalid state.

Be careful when do operation, make sure backup data if needed, and run vxdctl enable in non business time.

chuckchang23 · ‎05-09-2013

Hello Starflyfly,

Thank you for the input. I'm starting to understand what's going on :)

Since my goal is to reformat the disk and create partition, how do i make sure that files under the disk are not lost(if there are any?)

EMMDPD03:/# vxdisk list c0t2d0s2

Device: c0t2d0s2

devicetag: c0t2d0

type: auto

flags: error private autoconfig

pubpaths: block=/dev/vx/dmp/c0t2d0s2 char=/dev/vx/rdmp/c0t2d0s2

guid: -

udid: LSI%5FMR9262-8i%5FDISKS%5F600605B002952B10167068FF1DB9ECA3

site: -

Multipathing information:

numpaths: 1

c0t2d0s2 state=disabled

EMMDPD03:/#

pubpaths are already defined, is this ok?

starflyfly · ‎05-10-2013

Hi,

You should use format to add partition 2, with full disk capacity, then run vxdctl enable, it will change to "online invalid" state.

Here is my test log, hope you can first finish test in your bed.

================

root@hostAAA # prtvtoc /dev/rdsk/c2t500601603022FB3Fd17s2

* /dev/rdsk/c2t500601603022FB3Fd17s2 partition map

*

* Dimensions:

* 512 bytes/sector

* 10 sectors/track

* 64 tracks/cylinder

* 640 sectors/cylinder

* 32768 cylinders

* 32766 accessible cylinders

*

* Flags:

* 1: unmountable

* 10: read-only

*

* Unallocated space:

* First Sector Last

* Sector Count Sector

* 0 6400 6399

* 10492160 8389120 18881279

*

* First Sector Last

* Partition Tag Flags Sector Count Sector Mount Directory

3 9 01 6400 10485760 10492159

4 0 01 18881280 2088960 20970239 /test1

6 0 01 10492160 8389120 18881279 /test

7 15 01 0 20970240 20970239

root@hostAAA # vxdisk -eo alldgs list

DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR

emc_clariion0_37 auto - - error c2t500601603022FB3Fd17s2 lun RAID_5 <<<<

======

format

...

select disk

p

partition> p

Current partition table (unnamed):

Total disk cylinders available: 32766 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks

0 unassigned wu 0 0 (0/0/0) 0

1 unassigned wu 0 0 (0/0/0) 0

2 unassigned wu 0 0 (0/0/0) 0

3 alternates wu 10 - 16393 5.00GB (16384/0/0) 10485760

4 unassigned wu 29502 - 32765 1020.00MB (3264/0/0) 2088960

5 unassigned wu 0 0 (0/0/0) 0

6 unassigned wu 16394 - 29501 4.00GB (13108/0/0) 8389120

7 unassigned wu 0 0 (0/0/0) 0

partition> 2

Part Tag Flag Cylinders Size Blocks

2 unassigned wu 0 0 (0/0/0) 0

Enter partition id tag[unassigned]: backup

Enter partition permission flags[wu]:

Enter new starting cyl[0]:

Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: fie

Please specify units in either b(blocks), c(cylinders), e(end cylinder),

m(megabytes) or g(gigabytes)

Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 32766c<<<<last partition number

partition> p

Current partition table (unnamed):

Total disk cylinders available: 32766 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks

0 unassigned wu 0 0 (0/0/0) 0

1 unassigned wu 0 0 (0/0/0) 0

2 backup wu 0 - 32765 10.00GB (32766/0/0) 20970240

3 alternates wu 10 - 16393 5.00GB (16384/0/0) 10485760

4 unassigned wu 29502 - 32765 1020.00MB (3264/0/0) 2088960

5 unassigned wu 0 0 (0/0/0) 0

6 unassigned wu 16394 - 29501 4.00GB (13108/0/0) 8389120

7 unassigned wu 0 0 (0/0/0) 0

partition> label

Ready to label disk, continue?

Ready to label disk, continue? y

partition> q

FORMAT MENU:

disk - select a disk

type - select (define) a disk type

partition - select (define) a partition table

current - describe the current disk

format - format and analyze the disk

repair - repair a defective sector

label - write label to the disk

analyze - surface analysis

defect - defect list management

backup - search for backup labels

verify - read and display labels

save - save new disk/partition definitions

inquiry - show vendor, product and revision

volname - set 8-character volume name

!<cmd> - execute <cmd>, then return

quit

format> q

root@hostAAA # vxdisk list

DEVICE TYPE DISK GROUP STATUS

emc_clariion0_37 auto - - error <<<<<

root@hostAAA # vxdctl enable <<<<<

root@hostAAA # vxdisk list

DEVICE TYPE DISK GROUP STATUS

emc_clariion0_37 auto:none - - online invalid<<<<<

starflyfly · ‎05-10-2013

if you like do vxdisksetup for this disk, the data in the old file system will lost.

Yasuhisa_Ishika · ‎05-10-2013

I wonder why you tried to replace disk. There are no "failed" disk media in "vxdisk list" output. c0t2d0s2 is in error state, but it just mean c0t2d0s2 is not initialized at OS level for use of VxVM.

By prtvtoc output, c0t2d0s2 is used without VxVM, so do not format it by format command or vxdisksetup. Ignore 'error' status and keep it as it is.

Gaurav_S · ‎05-10-2013

Hi,

If a disk is in error state, the two main causes that drills down are:

1. disk partition table has issues

2. paths to disk are disable .

I agree with starflyfly's advice, you need to fix the partition table & get the disk to online invalid state. If the disk had data before & you want to recover data, you need to ensure that partition table is restored correctly as it was before & while intializing disk in veritas, you need to restore exsting configuration of privlen, publen attributes .. you can use this options with vxdisk init or vxdisksetup..

G

chuckchang23 · ‎05-13-2013

Hi Gaurav,

is there a way to check if the disk partition has files before?

kjbss · ‎05-14-2013

If you *know* that the disk used to be managed by VxVM, then you would need to get the partition layout as it was back then and reapply that exactly to the disk, as Gaurav says.

Interestingly, from the prtvtoc output you provided, partition 7 has the VxVM Tag (15) and points to the entire disk, so in that way it does appear that it was at one time a VxVM managed disk.

However, the prtvtoc output also now shows two slices (4 and 6) with Mount Directories associated with them, and so it looks like this disk was *last* used as a test disk outside of VxVM management:

*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       3      9    01       6400  10485760  10492159
       4      0    01   18881280   2088960  20970239   /test1
       6      0    01   10492160   8389120  18881279   /test
       7     15    01          0  20970240  20970239

We can see that slices 4 and 6 were likely used for testing purposes, but if you really want to see the files in those partitions:

...Taking slice 6 as an example:

Check to see if it is a mountable partition:
$ fstyp /dev/rdsk/c2t500601603022FB3Fd17s6

If it shows to be a filesystem on there, like ufs or vxfs, sanity-check the filesystem:
$ fsck -F <FStype> -y /dev/rdsk/c2t500601603022FB3Fd17s6

Make a temporary mount point for it:
$ mkdir /tmp_test

Mount it:
$ mount -F <FStype> /dev/dsk/c2t500601603022FB3Fd17s6 /tmp_test

Inspect it however you want to...
$ ls -l /tmp_test
...etc...

kjbss · ‎05-14-2013

Hi again chuckchang23 --

I just noticed I based my last advice to you on the prtvtoc output from starflyfly's test envrionment!

Naughty of me; sorry about that...

The pertinent section of your 'prtvtoc /dev/rdsk/c0t2d0s2' output shown below:

*                          First     Sector    Last
* Partition Tag Flags    Sector     Count    Sector Mount Directory
       3      9    00      16065 10490445 10506509   /opt/bmc
       6     11    00   10506510 573424110 583930619   /backup
       8      1    01          0     16065     16064

Well, it would appear to me that this disk never had VxVM management on it, or at least has been reformatted since.

The point is, if you want to see the data on there...:

(Using slice 3 as the expample, which seems to contain BMC software data (/opt/bmc) data on there)...

Check to see if it is a mountable partition:
$ fstyp /dev/rdsk/c0t2d0s3

If it shows to be a filesystem on there, like ufs or vxfs, sanity-check the filesystem:
$ fsck -F <FStype> -y /dev/rdsk/c0t2d0s3

Make a temporary mount point for it:
$ mkdir /tmp_opt_bmc

Mount it:
$ mount -F <FStype> /dev/dsk/c0t2d0s3 /tmp_opt_bmc

Inspect it however you want to...
$ ls -l /tmp_opt_bmc
...etc...

chuckchang23 · ‎05-21-2013

Hi kjbss,

The first command issued an error on my faulted disk

EMMDPD03:/# fstyp /dev/rdsk/c0t2d0s2

hsfs fstyp: cannot open </dev/rdsk/c0t2d0s2>

/dev/rdsk/c0t2d0s2: I/O error

udfs fstyp: cannot open </dev/rdsk/c0t2d0s2> errorno <5>

/dev/rdsk/c0t2d0s2: I/O error

UX:vxfs fstyp: ERROR: V-3-20003: Cannot open /dev/rdsk/c0t2d0s2: I/O error

open64: I/O error

Unknown_fstyp (no matches)

EMMDPD03:/#

EMMDPD03:/# fsck -F ufs -y /dev/rdsk/c0t2d0s2

Can't open /dev/rdsk/c0t2d0s2: I/O error

EMMDPD03:/#

stinsong · ‎05-21-2013

Hi chuck,

The I/O error indicate that there is I/O problem with the disk /dev/rdsk/c0t2d0s2.

You should check on the I/O error first for HW issue or link problem.

kjbss · ‎05-24-2013

chuckchange23 -

It would seem that you have transient access to disk /dev/rdsk/c0t2d0s2, because on your 10 May 2013 post above, you show 'prtvtoc /dev/rdsk/c0t2d0s2' working -- this would not have worked if you had I/O errors to that disk, as shown in your 22 May 2013 post.

As stinsong points out, you need to investigate and resolve your I/O errors first.

Do you have this drive on an array that has active/passive support to your host? Maybe another host has the active path to the drive, or some other kind of SAN zoning preventing you from accessing that drive from your EMMDPD03 host on the 22 May.

VOX

Vxdiskadm Replace disk does not show my failed disk