VxVM won't play with replaced SCSI disk

[harold00]

Hi, sure seems I'm missing something(s) easy and/or obvious, but I have tried a lot of stuff and am getting nowhere...

Platform is a very legacy Sun E2900 with dual SCSI internal disks and FC to an external array, with Veritas VM in charge of all the disk.  Problem is with the internals, which are/were configured as a mirrored rootdg pair.  One of the disks died and was pysically replaced, and seems to be fine (green light is now on, and cfgadm -a says connected and configured).  Nothing was done with Veritas up to this point (and maybe that's the cause of the trouble?!).

The new drive seems functional, can do format and prtvtoc operations, dd data from it and all, but none of the vx commands will deal, claiming it doesn't exist:

# vxdisksetup c1t1d0 format=sliced

prtvtoc: /dev/vx/rdmp/c1t1d0s2: No such device or address

and the state according to vxdisk list is just "error":

# vxdisk list c1t1d0

Device:    c1t1d0s2

devicetag: c1t1d0

type:      auto

flags:     online error private autoconfig

errno:     Device path not valid

Multipathing information:

numpaths:   1

c1t1d0s2        state=enabled

Have tried removing the device files in /dev/vx/rdmp, followed up with "vxdctl initdmp", and a bunch of other stuff found on fora like this one, but to no avail. Thanks for any bread crumbs in advance!

[Gaurav_S]

Hello,

please mention OS version & VxVM version as well..

In case you are using VxVM 4.x  ,I would suggest you to have device tree cleanup (main step to remove disk.info file)

you can refer to below article:

http://www.symantec.com/docs/TECH76780

From the above article, you can skip the emc powerpath related activities as the issue lies with internal disk...

Once you execute above steps, I would expect the new disk to be visible in "vxdisk list" output with a state of "online invalid"..

G

[Dev_Roy]

Can you please try following commands:

devfsadm -Cv

vxdctl enable

Then try initializing the disk once again. If this does not help, please let us know following command outputs:

vxdisk -o alldgs -e list

vxprint -qhtrg rootdg

modinfo | grep -i vx

uname -a

Also checkout if you have stale device file entries in your system, if so a device tree clean up may be suggested.

Regards,

Dev

[Anoop_Kumar1]

Please paste below commands outputs.

- modinfo | grep vx

- vxdisk -o alldgs -e list

- prtvtoc

[jstucki]

Your best bet at this point is to open a Support Case, and have a Symantec Support Engineer guide you through the steps to get the new disk working with VxVM and synced with the remaining mirror.

It would have been a fairly easy process had you used the vxdiskadm utility to do all the work for you.  If I remember correctly, you use an option in vxdiskadm to tell VxVM that you're going to replace the failed disk.  Then you physically replace it.  Then you use another option in vxdiskadm to perform all the steps to sync the new disk up with the remaining mirror.  It does everything for you.  But I think its going to be several manual steps now.  Best to work with Support.

[harold00]

Wow thanks for all the quick replies.  Sorry, OS is Solaris 5.9 and VxVM appears to be 4.0.  Tried the device cleanup procedure, but no change.  Here are the outputs that were requested.  Tx again.

# uname -a

SunOS prdrms01 5.9 Generic_122300-54 sun4u sparc SUNW,Netra-T12

# vxdisk -o alldgs -e list

DEVICE       TYPE      DISK         GROUP        STATUS       OS_NATIVE_NAME

c1t0d0s2     auto      rootdg01      rootdg       online       c1t0d0s2

c1t1d0s2     auto      -             -            error        c1t1d0s2

c1t1d0s2     auto      -             -            error        c1t1d0s2

c1t1d0s2     auto      -             -            error        c1t1d0s2

[deleted out a long list of unaffected FC disks]

-            -         rootdg02     rootdg       removed was:c1t1d0s2

# prtvtoc /dev/rdsk/c1t1d0s2

* /dev/rdsk/c1t1d0s2 partition map

*

* Dimensions:

*     512 bytes/sector

*     424 sectors/track

*      24 tracks/cylinder

*   10176 sectors/cylinder

*   14089 cylinders

*   14087 accessible cylinders

*

* Flags:

*   1: unmountable

*  10: read-only

*

*                          First     Sector    Last

* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory

       0      2    00          0    264576    264575

       1      3    01     264576    264576    529151

       2      5    01          0 143349312 143349311

       6      4    00     529152 142820160 143349311

# vxprint -qhtrg rootdg

dg rootdg       default      default  17000    1106418263.6.prdrms01

dm rootdg01     c1t0d0s2     auto     20351    143349311 -

dm rootdg02     -            -        -        -        REMOVED

sd rootdg01Priv -            rootdg01 127841088 20351   PRIVATE   c1t0d0   ENA

v  globaldevices -           ENABLED  ACTIVE   529152   ROUND     -        fsgen

pl globaldevices-01 globaldevices ENABLED ACTIVE 529152 CONCAT    -        RW

sd rootdg01-04  globaldevices-01 rootdg01 142291007 529152 0      c1t0d0   ENA

pl globaldevices-02 globaldevices DISABLED REMOVED 529152 CONCAT  -        RW

sd rootdg02-04  globaldevices-02 rootdg02 142270656 529152 0      -        RMOV

v  rootdg017vol -            ENABLED  ACTIVE   529152   ROUND     -        gen

pl rootdg017vol-01 rootdg017vol ENABLED ACTIVE 529152   CONCAT    -        RW

sd rootdg01-03  rootdg017vol-01 rootdg01 142820159 529152 0       c1t0d0   ENA

pl rootdg017vol-02 rootdg017vol DISABLED REMOVED 529152 CONCAT    -        RW

sd rootdg02-05  rootdg017vol-02 rootdg02 142799808 529152 0       -        RMOV

v  rootvol      -            ENABLED  ACTIVE   127841088 ROUND    -        root

pl rootvol-01   rootvol      ENABLED  ACTIVE   127841088 CONCAT   -        RW

sd rootdg01-B0  rootvol-01   rootdg01 127841087 1       0         c1t0d0   ENA

sd rootdg01-02  rootvol-01   rootdg01 0        127841087 1        c1t0d0   ENA

pl rootvol-02   rootvol      DISABLED REMOVED  127841088 CONCAT   -        RW

sd rootdg02-01  rootvol-02   rootdg02 0        127841088 0        -        RLOC

v  swapvol      -            ENABLED  ACTIVE   4182336  ROUND     -        swap

pl swapvol-01   swapvol      ENABLED  ACTIVE   4182336  CONCAT    -        RW

sd rootdg01-01  swapvol-01   rootdg01 127861439 4182336 0         c1t0d0   ENA

pl swapvol-02   swapvol      DISABLED REMOVED  4182336  CONCAT    -        RW

sd rootdg02-02  swapvol-02   rootdg02 127841088 4182336 0         -        RMOV

v  var          -            ENABLED  ACTIVE   10247232 ROUND     -        fsgen

pl var-01       var          ENABLED  ACTIVE   10247232 CONCAT    -        RW

sd rootdg01-05  var-01       rootdg01 132043775 10247232 0        c1t0d0   ENA

pl var-02       var          DISABLED REMOVED  10247232 CONCAT    -        RW

sd rootdg02-03  var-02       rootdg02 132023424 10247232 0        -        RLOC

# modinfo|grep -i vx

 17  11d616a 1c8d20 275   1  vxio (VxVM 4.0R_p3.7 (MP1) I/O driver)

 33 782a4000  27f9c 274   1  vxdmp (VxVM 4.0R_p3.7 (MP1): DMP Drive)

 34 782c81d4    847 276   1  vxspec (VxVM 4.0R_p3.7 (MP1) control/st)

283 7869d30f    bb3 277   1  vxportal (VxFS 4.0_REV-MP1j portal driver)

284 7870c000 163aad   8   1  vxfs (VxFS 4.0_REV-MP1j,PID=116689-01)

285 78860000   a169 278   1  fdd (VxQIO 4.0_REV-MP1j Quick I/O dr)

287 78976000  188e7 279   1  qlog (VxQLOG 4.0_REV-MP1j QuickLog dr)

[harold00]

Thanks, good idea but, I'm sure any support contract we may have had expired years ago.  Purely desperation time here...

[Marianne]

No more support in any case for 4.0....

Lets try again to cleanup the device tree. This TN  says to also cleanup the /dev/dsk and /dev/rdsk (except for c0t0 or whatever your current boot disk is...)

http://www.symantec.com/docs/TECH57018

I also found in old case notes reference to darecs file in /etc/vx that apparently existed in 4.x...

So, add that when you rename files in /etc/vx/.

[Gaurav_S]

One more issue I see over here is duplicate devices:

c1t1d0s2     auto      -             -            error        c1t1d0s2

c1t1d0s2     auto      -             -            error        c1t1d0s2

c1t1d0s2     auto      -             -            error        c1t1d0s2

though to take care of this issue, many of technotes suggest the same device tree cleanup procedure what I recommended before however since this is 4.0, I am presuming you might need a reconfiguration reboot ..

If the suggestion above doesn't help, I would suggest to do a reconfig reboot ..

Below technote explains the same with SF 4.1

http://www.symantec.com/docs/TECH129831

Gaurav

[Andrew_J]

I have a two node Veritas CFS cluster. DR is provided by a remote standalone node that only runs VXVM and VXFS. For the purposes of DR, it does currently serve its purpose. However, There is a mismatch in the s/w versions. I do realise these versions have been superseeded. However, I want to get the DR system at exactly the same revision(s)..prior to planning an upg...

What is the recommended upg path/patch set ?? Many thanks..

From MP1_RP4 to MP3_RP2

Many thanks

Production

29  13a6958  3eea0 291   1  vxdmp (VxVM 5.0MP3RP2: DMP Driver)

31 7be00000 20bbf0 292   1  vxio (VxVM 5.0MP3RP2 I/O driver)

33 7bfeb508    c78 293   1  vxspec (VxVM 5.0MP3RP2 control/status d)

215 7af88000  521d8 290   1  vxfen (VRTS Fence 5.0MP3RP2)

216 7af874d8    cb0 294   1  vxportal (VxFS 5.0_REV-5.0MP3RP2f_sol por)

217 7a600000 1dcc00  21   1  vxfs (VxFS 5.0_REV-5.0MP3RP2f_sol Sun)

218 7afba000  21ec0 296   1  vxglm (VxGLM 5.0MP3 (SunOS 5.10))

236 7a7cc000   ab30 295   1  fdd (VxQIO 5.0_REV-5.0MP3RP2f_sol Qu)

DR Has :

STANDALONE DR CLUSTER

29  13a6958  3aed8 291   1  vxdmp (VxVM 5.0_MP1_RP4_HF10: DMP Driv)

31 7c002000 3409c8 292   1  vxio (VxVM 5.0_MP1_RP4_HF10 I/O drive)

33  13dd9a0    d48 293   1  vxspec (VxVM 5.0_MP1_RP4_HF10 control/s)

183 7be77300    c38 294   1  vxportal (VxFS 5.0_REV-5.0MP1RP3a3_sol po)

184 7ae00000 1bd048  21   1  vxfs (VxFS 5.0_REV-5.0MP1RP3a3_sol Su)

230 7beea000  4fe28 290   1  vxfen (VRTS Fence 5.0MP1)

247 7ab58000   a2c8 295   1  fdd (VxQIO 5.0_REV-5.0MP1RP3a3_sol Q)