Forum Discussion

Rob74's avatar
Rob74
Level 4
13 years ago

Double disk failure, recovering rootdg

Through no fault of mine, angel upon replacing rootmirror, rootdisk went belly up in a V880.  we booted off DVD, placed an OS on disk1, installed netbackup software, and restored through disk1 to disk0 which was on /mnt, /mnt/usr, and /mnt/var.  Restore worked fine, we followed some guidelines, touched the install-db file, and rebooted.  when I attempted to re-encapsulate the disk that failed but now I have a rootdg I cant get rid of.  How can I tell veritas this is a clean disk, and re-encapsulate the disk for adding to the original rootdg?  or importing the old rootdg to a new DG and then moving those volumes to the new rootdg?  I was thinking I could add c3t1d0s2 to rootdg, remove c3t0d0s2 and then run vxdiskadm on c3t0d0s2 to encapsulate it.  I can do another restore if need be to disk0.  We were trying to follow the following, however there was no bootdg so we did not perform step 15. using vxedit I was able to remove all the various plexes, subdisks, and volumes but I can not remove remove the new disk group or disk. https://sort.symantec.com/public/documents/sf/5.0/solaris/html/vxvm_tshoot/ts_ch_Solarisbootdiskrecovery_vm15.html

Bear in mind, this is part of a cluster, and the cluster appears to be running ok still.  I am hoping to nurse this back to life and run for another year or so.  also yes I am aware the webip is faulted, and yes I know this stuff was EOL like years ago.

garbcorp1# pkginfo -l VRTSvmsa
   PKGINST:  VRTSvmsa
      NAME:  VERITAS Volume Manager Storage Administrator
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  3.2,REV=07.27.2001.19.47
 

garbcorp1# rm /etc/vx/reconfig.d/state.d/install-db
garbcorp1# vxiod set 10
garbcorp1# vxconfigd -m disable
garbcorp1# vxdctl enable
vxvm:vxconfigd: WARNING: Disk c3t5d0s2 names group rootdg, but group ID differs
vxvm:vxconfigd: WARNING: Disk c3t2d0s2 names group rootdg, but group ID differs
vxvm:vxconfigd: WARNING: Disk c3t4d0s2 names group rootdg, but group ID differs
vxvm:vxconfigd: WARNING: Disk c3t3d0s2 names group rootdg, but group ID differs


garbcorp1# vxdg -f destroy rootdg
vxvm:vxdg: ERROR: Disk group rootdg: deport failed: Operation not allowed on root disk group
garbcorp1# vxdg -o override rmdisk rootdisk
vxvm:vxdg: ERROR: disassociating disk-media rootdisk:
        Cannot remove last disk in disk group
garbcorp1# vxdisk -o alldgs list
DEVICE       TYPE      DISK         GROUP        STATUS
c1t0d0s2     sliced    -           (prod-dg1)    online
c1t1d0s2     sliced    -           (prod-dg1)    online
c1t2d0s2     sliced    -           (prod-dg1)    online
c1t3d0s2     sliced    -           (prod-dg2)    online
c1t4d0s2     sliced    -           (prod-dg2)    online
c1t5d0s2     sliced    -           (prod-dg2)    online
c1t8d0s2     sliced    -           (ingres-dg1)  online
c1t9d0s2     sliced    -           (ingres-dg1)  online
c1t10d0s2    sliced    -           (ingres-dg1)  online
c1t11d0s2    sliced    -           (ingres-dg2)  online
c1t12d0s2    sliced    -           (ingres-dg2)  online
c1t13d0s2    sliced    -           (ingres-dg2)  online
c2t0d0s2     sliced    -           (prod-dg1)    online
c2t1d0s2     sliced    -           (prod-dg1)    online
c2t2d0s2     sliced    -           (prod-dg1)    online
c2t3d0s2     sliced    -           (prod-dg2)    online
c2t4d0s2     sliced    -           (prod-dg2)    online
c2t5d0s2     sliced    -           (prod-dg2)    online
c2t8d0s2     sliced    -           (ingres-dg1)  online
c2t9d0s2     sliced    -           (ingres-dg1)  online
c2t10d0s2    sliced    -           (ingres-dg1)  online
c2t11d0s2    sliced    -           (ingres-dg2)  online
c2t12d0s2    sliced    -           (ingres-dg2)  online
c2t13d0s2    sliced    -           (ingres-dg2)  online
c3t0d0s2     sliced    rootdisk     rootdg       online
c3t1d0s2     sliced    -            -            online
c3t2d0s2     sliced    -           (rootdg)      online
c3t3d0s2     sliced    -           (rootdg)      online
c3t4d0s2     sliced    -           (rootdg)      online
c3t5d0s2     sliced    -           (rootdg)      online
garbcorp1# vxprint -qhtrg rootdg
dg rootdg       default      default  0        1343344112.1025.garbcorp1

dm rootdisk     c3t0d0s2     sliced   2888     71124291 -

garbcorp1# vxdg list rootdg
Group:     rootdg
dgid:      1343344112.1025.garbcorp1
import-id: 0.1
flags:   
version:   90
detach-policy: global
copies:    nconfig=default nlog=default
config:    seqno=0.1407 permlen=2112 free=2110 templen=2 loglen=320
config disk c3t0d0s2 copy 1 len=2112 state=clean online
log disk c3t0d0s2 copy 1 len=320
 

garbcorp1# /opt/VRTSvcs/bin/hastatus -summary

-- SYSTEM STATE
-- System               State                Frozen             

A  garbcorp1            RUNNING              0                   
A  garbcorp2            RUNNING              0                   

-- GROUP STATE
-- Group           System               Probed     AutoDisabled    State         

B  ClusterService  garbcorp1            Y          N               ONLINE        
B  ClusterService  garbcorp2            Y          N               OFFLINE|FAULTED
B  production_grp  garbcorp1            Y          N               OFFLINE       
B  production_grp  garbcorp2            Y          N               ONLINE        

-- RESOURCES FAILED
-- Group           Type                 Resource             System             

   C  ClusterService  IP                   webip                garbcorp2          

business continuity
Clustering
Configuring
disaster recovery
Recovering
Solaris
Storage Foundation
  • Rob74's avatar
    Rob74
    13 years ago

    since I could not delete rootdg I added rootmirror disk to the group, reencapsulated rootdisk, mirrored with option 6, and then readded my other rootdg disks.  I seem to be okay now

  • TonyGriffiths's avatar
    TonyGriffiths
    Level 6
    13 years ago

    Hi

    As you mention, 3.2 is a v.old version that is EOSL. Versions prior to 4.x (also EOSL) had a requirement that there must be a disk group named rootdg for vxvm to start.

    In your output above, there seems to be a valid rootdg, with one disk (c3t0d0) and no volumes. VxVM appear to be functioning.

    There are other disks that appear to have been in rootdg at some stage, but its likely that VxVM is using the most recently created one.

    Q>Which disk (CTD name )is the boot disk and is your intent to encapsulate the boot disk into vxvm ?

    Q>When you restored the data to create the new boot disk, did it place any original partitions that may indicate it was beloging to vxvm (prtvtoc output will help)

    cheers

    tony

  • Rob74's avatar
    Rob74
    Level 4
    13 years ago

    c3t0d0s2 is the original boot disk

    c3t1d0s2 is the boot disk we placed an OS on to boot from and added netbackup to restore (we couldnt just boot from the DVD media as the netbackup client isnt on it smiley)

    we restored the partitions but recieved a failure trying to re-encapsulate it

    you can not see partitions during a restore it is whatever partitions you create, home appears to have been created by the vxvol command.  I redid the partitioning except the encasulation partitions newfs'd and remounted, and restored all but /export/home.  during the reboot it could not find an /etc/system (oops) and when it went to add the rootdev info to /etc/system it failed and now it's kinda in a limbo. 

    0=/

    1=swap

    5=/usr

    6=/var

    garbcorp1# prtvtoc /dev/rdsk/c3t0d0s2
    * /dev/rdsk/c3t0d0s2 partition map
    *
    * Dimensions:
    *     512 bytes/sector
    *     107 sectors/track
    *      27 tracks/cylinder
    *    2889 sectors/cylinder
    *   24622 cylinders
    *   24620 accessible cylinders
    *
    * Flags:
    *   1: unmountable
    *  10: read-only
    *
    * Unallocated space:
    *       First     Sector    Last
    *       Sector     Count    Sector
    *    12584484 4282382812 4294967295
    *    71127180 4236424600  12584483
    *    41948280  29176011  71124290
    *
    *                          First     Sector    Last
    * Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
           0      2    00          0  12584484  12584483   /
           1      3    01   12584484  12584484  25168967
           2      5    00          0  71127180  71127179
           3     14    01          0  71127180  71127179
           4     15    01   71124291      2889  71127179
           5      4    00   25168968   8389656  33558623   /usr
           6      7    00   33558624   8389656  41948279   /var
     

  • Rob74's avatar
    Rob74
    Level 4
    13 years ago

    Also, I have added c3t1d0s2 as rootmirror to rootdg, and removed c3t0d0s2 from rootdg, I am in the process of re-encapsulating it, however I will not be able to reboot the node for another 6.5 hours, to find out if it is sucessful.

  • Rob74's avatar
    Rob74
    Level 4
    13 years ago

    That seems to have worked, I have now created a home volume.

    garbcorp1# vxdg rmdisk rootmirror
    garbcorp1# vxassist -g rootdg make home 12g
    garbcorp1# vxassist -g rootdg maxgrow home
    Volume home can be extended by 4007936 to 29173760 (14245Mb)
    garbcorp1# vxassist -g rootdg growby home 4007936
     

    I have now used vxdiskadm

    chose option #1, and added c3t1d0 to rootdg, and named rootmirror

    and then option #6 to mirror volumes to rootmirror,

    I will be running the following to take ownership of the cluster to confirm I am good to go...

    garbcorp1# /opt/VRTSvcs/bin/hagrp -switch production_grp -to garbcorp1
     

    enlightened

  • Rob74's avatar
    Rob74
    Level 4
    13 years ago

    since I could not delete rootdg I added rootmirror disk to the group, reencapsulated rootdisk, mirrored with option 6, and then readded my other rootdg disks.  I seem to be okay now

  • TonyGriffiths's avatar
    TonyGriffiths
    Level 6
    13 years ago

    Hi,

    Its sounds like you have sorted it, though one thing is puzzling me.

    You mentioned that c3t1d0 is the new boot disk (restored data etc) and that this has been added into root dg as rootmir.

    Then you mention that you mirrored the volumes to rootmirr, does this mean that you mirrored the data from a different disk TO c3t1d0 or you mirrored data from c3t1d0 to another disk (rootdisk?)

     

    just a thought

    cheers

    tony