Veritas Volume Manager problem

[gregsih]

We have 2 Sun 3510 arrays each containing three disks. VxVM, on Solaris 10 for SPARC, has been set up to mirror each disk, ie Array 1 Disk 1 is mirrored to Array 2 Disk 1 etc etc. We had a problem with the power to the controller of array 2 and the disks in that array ended up in a disabled and removed state.

The power has now been restored and the disks are now visible to the operating system again. However whatever we try we can not get VxVM to accept the disks.

It can see them but will not re-add them. We have followed the replace disk procedure which fails with the message 'no device available to replace'.

Any suggestions would be most gratefuly received.

[gregsih]

Hi Grace

Sorry but same result :(

Regards

Greg

[g_lee]

Greg,

Please provide this output to confirm the current state of the dg:

# vxprint -qhtrg datadg

In the meantime I'll put together the commands to remove the old dg config from the disks and add them in as fresh disks (since you'll have to resync the mirrors from scratch anyway) - not ideal but seems to be necessary due to the way the disks were removed.

regards,
Grace

[gregsih]

Hi Grace

Please see below

Regards

Greg

root@server01 # vxprint -qhtrg datadg
dg datadg       default      default  41000    1129915301.61.dwprod01

dm datadg01     Disk_4       auto     2048     1140796160 -
dm datadg02     Disk_5       auto     2048     1140796160 -
dm datadg03     Disk_6       auto     2048     1140796160 -
dm datadg04     -            -        -        -        REMOVED
dm datadg05     -            -        -        -        REMOVED
dm datadg06     -            -        -        -        REMOVED

v  vol01        -            ENABLED  ACTIVE   1140795392 SELECT  -        fsgen
pl vol01-01     vol01        ENABLED  ACTIVE   1140795392 CONCAT  -        RW
sd datadg01-01  vol01-01     datadg01 0        1140795392 0       Disk_4   ENA
pl vol01-02     vol01        DISABLED REMOVED  1140795392 CONCAT  -        WO
sd datadg04-01  vol01-02     datadg04 0        1140795392 0       -        RMOV

v  vol02        -            ENABLED  ACTIVE   1140795392 SELECT  -        fsgen
pl vol02-01     vol02        ENABLED  ACTIVE   1140795392 CONCAT  -        RW
sd datadg02-01  vol02-01     datadg02 0        1140795392 0       Disk_5   ENA
pl vol02-02     vol02        DISABLED REMOVED  1140795392 CONCAT  -        WO
sd datadg05-01  vol02-02     datadg05 0        1140795392 0       -        RMOV

v  vol03        -            ENABLED  ACTIVE   1140795392 SELECT  -        fsgen
pl vol03-01     vol03        ENABLED  ACTIVE   1140795392 CONCAT  -        RW
sd datadg03-01  vol03-01     datadg03 0        1140795392 0       Disk_6   ENA
pl vol03-02     vol03        DISABLED REMOVED  1140795392 CONCAT  -        WO
sd datadg06-01  vol03-02     datadg06 0        1140795392 0       -        RMOV

[g_lee · Accepted solution]

Greg,

Try on Disk_9 first

# vxdiskunsetup -C Disk_9

# vxdisksetup -i Disk_9 format=cdsdisk privlen=2048

# vxdisk list Disk_9

The fields in bold (from your previous output) should be the same. If they're not, stop here and post the vxdisk list Disk_9 output!

root@server01 # vxdisk list Disk_9
[...]
info:      format=cdsdisk,privoffset=256,pubslice=2,privslice=2
flags:     online ready private autoconfig autoimport
pubpaths:  block=/dev/vx/dmp/Disk_9s2 char=/dev/vx/rdmp/Disk_9s2
version:   3.1
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=2 offset=2304 len=1140796160 disk_offset=0
private:   slice=2 offset=256 len=2048 disk_offset=0
[...]
Multipathing information:
numpaths:   1
c5t40d1s2       state=enabled

If they are the same, try to add this to the dg now:

# vxdg -g datadg -k adddisk datadg04=Disk_9

Repeat steps for Disk_10, Disk_11 (ensure privlen=2048 is specified in the vxdisksetup command)

If this works, then post the vxprint -qhtrg datadg output after adding the disks so the plexes can be recovered

[gregsih]

Grace

Output below.

Regards

Greg

root@server01 # vxprint -qhtrg datadg
dg datadg       default      default  41000    1129915301.61.dwprod01

dm datadg01     Disk_4       auto     2048     1140796160 -
dm datadg02     Disk_5       auto     2048     1140796160 -
dm datadg03     Disk_6       auto     2048     1140796160 -
dm datadg04     Disk_9       auto     2048     1140796160 NOHOTUSE
dm datadg05     Disk_10      auto     2048     1140796160 -
dm datadg06     Disk_11      auto     2048     1140796160 NOHOTUSE

v  vol01        -            ENABLED  ACTIVE   1140795392 SELECT  -        fsgen
pl vol01-01     vol01        ENABLED  ACTIVE   1140795392 CONCAT  -        RW
sd datadg01-01  vol01-01     datadg01 0        1140795392 0       Disk_4   ENA
pl vol01-02     vol01        DISABLED RECOVER  1140795392 CONCAT  -        WO
sd datadg04-01  vol01-02     datadg04 0        1140795392 0       Disk_9   ENA

v  vol02        -            ENABLED  ACTIVE   1140795392 SELECT  -        fsgen
pl vol02-01     vol02        ENABLED  ACTIVE   1140795392 CONCAT  -        RW
sd datadg02-01  vol02-01     datadg02 0        1140795392 0       Disk_5   ENA
pl vol02-02     vol02        DISABLED RECOVER  1140795392 CONCAT  -        WO
sd datadg05-01  vol02-02     datadg05 0        1140795392 0       Disk_10  ENA

v  vol03        -            ENABLED  ACTIVE   1140795392 SELECT  -        fsgen
pl vol03-01     vol03        ENABLED  ACTIVE   1140795392 CONCAT  -        RW
sd datadg03-01  vol03-01     datadg03 0        1140795392 0       Disk_6   ENA
pl vol03-02     vol03        DISABLED RECOVER  1140795392 CONCAT  -        WO
sd datadg06-01  vol03-02     datadg06 0        1140795392 0       Disk_11  ENA

[g_lee]

Greg,

That looks good - the disks are back in the diskgroup.

This should recover the disabled plexes:

# vxrecover -g datadg -sb

Check:

# vxtask list

# vxprint -qhtrg datadg

[gregsih]

Hi Grace

That all worked fine and the plex's are recovering now.

May I say many thanks to you and all of the other contributors for all of your help and patience with a VxVM novice.

Once again many thanks to you all.

Regards

Greg

[g_lee]

Greg,

Glad to hear the problem has been resolved now.

Just to reiterate - for future disk replacements/recovery, try using vxdiskadm - option 4 (Remove a disk for replacement) as the recovery should be more straightforward (should be able to use vxdiskadm option 5, or vxreattach as suggested in the earlier posts, rather than having to reinitialise disks to put them back in!)

regards,

Grace