Solved: Dynamic multipath using EMC storage Veritas 3.5 - Page 2

devans3428 · ‎04-06-2011

I am trying to setup multipathing with EMC Clariion. The problem is that vxdisk list fabric_0 only shows one path. The EMC array is in auto-trespass mode. This is solaris 8 and format shows two paths.

# vxdisk list fabric_2
Device:    fabric_2
devicetag: fabric_2
type:      sliced
hostid:    ncsun1
disk:      name=disk05 id=1302111549.6037.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
info:      privoffset=1
flags:     online ready private autoconfig autoimport imported
pubpaths: block=/dev/vx/dmp/fabric_2s4 char=/dev/vx/rdmp/fabric_2s4
privpaths: block=/dev/vx/dmp/fabric_2s3 char=/dev/vx/rdmp/fabric_2s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1048494080
private:   slice=3 offset=1 len=32511
update:    time=1302111558 seqno=0.5
headers:   0 248
configs:   count=1 len=23969
logs:      count=1 len=3631
Defined regions:
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
log      priv 023987-027617[003631]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c10t500601613B241045d5s2        state=enabled

formt

8. c10t500601613B241045d0 <DGC-RAID5-0428 cyl 63998 alt 2 hd 256 sec 64>
/ssm@0,0/pci@19,700000/SUNW,qlc@2/fp@0,0/ssd@w500601613b241045,0

16. c16t500601603B241045d0 <DGC-RAID5-0428 cyl 63998 alt 2 hd 256 sec 64>
/ssm@0,0/pci@18,700000/SUNW,qlc@1/fp@0,0/ssd@w500601603b241045,0

vxdisk -o alldgs list show both paths. Two things here it should only show one of the paths and also the second path it shows with diskgroup in ( ). Another issue is why the disk dont show up as EMC_0 or similiar.

*****The server has T3 connect and EMC which we are migrating from T3 to EMC. The EMC is the fabric naming convention.

# vxdisk -o alldgs list
DEVICE       TYPE      DISK         GROUP        STATUS
T30_0        sliced    disk01       rootdg       online
T30_1        sliced    disk02       rootdg       online
T31_0        sliced    disk03       rootdg       online
T31_1        sliced    disk04       rootdg       online
T32_0        sliced    rootdg00     rootdg       online
T32_1        sliced    rootdg01     rootdg       online
c1t0d0s2     sliced    -            -            error
c1t1d0s2     sliced    -            -            error
fabric_0     sliced    -            -            error
fabric_1     sliced    -            -            error
fabric_2     sliced    disk05       rootdg       online
fabric_3     sliced    disk06       rootdg       online
fabric_4     sliced    disk07       rootdg       online
fabric_5     sliced    disk08       rootdg       online
fabric_6     sliced    disk09       rootdg       online
fabric_7     sliced    disk10       rootdg       online
fabric_8     sliced    -            -            error
fabric_9     sliced    -            -            error
fabric_10    sliced    -           (rootdg)      online
fabric_11    sliced    -           (rootdg)      online
fabric_12    sliced    -           (rootdg)      online
fabric_13    sliced    -           (rootdg)      online
fabric_14    sliced    -           (rootdg)      online
fabric_15    sliced    -           (rootdg)      online

Here is the ASL...There is no APM prior to version Veritas 4.0.

vxddladm listsupport

snippet for brevity.....

libvxDGCclariion.so A/P DGC CLARiiON

The c10 and c16 are the paths for the EMC

# vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c1              OTHER_DISKS     ENABLED      OTHER_DISKS
c10             OTHER_DISKS     ENABLED      OTHER_DISKS
c16             OTHER_DISKS     ENABLED      OTHER_DISKS

# vxdmpadm getsubpaths ctlr=c10
NAME         STATE         PATH-TYPE DMPNODENAME ENCLR-TYPE   ENCLR-NAME
======================================================================
c10t500601613B241045d7s2 ENABLED        -        fabric_0     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d6s2 ENABLED        -        fabric_1     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d5s2 ENABLED        -        fabric_2     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d4s2 ENABLED        -        fabric_3     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d3s2 ENABLED        -        fabric_4     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d2s2 ENABLED        -        fabric_5     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d1s2 ENABLED        -        fabric_6     OTHER_DISKS OTHER_DISKS
c10t500601613B241045d0s2 ENABLED        -        fabric_7     OTHER_DISKS OTHER_DISKS
# vxdmpadm getsubpaths ctlr=c16
NAME         STATE         PATH-TYPE DMPNODENAME ENCLR-TYPE   ENCLR-NAME
======================================================================
c16t500601603B241045d7s2 ENABLED        -        fabric_8     OTHER_DISKS OTHER_DISKS
c16t500601603B241045d6s2 ENABLED        -        fabric_9     OTHER_DISKS OTHER_DISKS
c16t500601603B241045d5s2 ENABLED        -        fabric_10    OTHER_DISKS OTHER_DISKS
c16t500601603B241045d4s2 ENABLED        -        fabric_11    OTHER_DISKS OTHER_DISKS
c16t500601603B241045d3s2 ENABLED        -        fabric_12    OTHER_DISKS OTHER_DISKS
c16t500601603B241045d2s2 ENABLED        -        fabric_13    OTHER_DISKS OTHER_DISKS
c16t500601603B241045d1s2 ENABLED        -        fabric_14    OTHER_DISKS OTHER_DISKS
c16t500601603B241045d0s2 ENABLED        -        fabric_15    OTHER_DISKS OTHER_DISKS

Thanks for any help

g_lee · ‎04-08-2011

Strange - in your original post, the disk with dmname disk01 was T30_0 (ie: T3 disk), but now this is the only disk showing as imported (and there should only be EMC disks left?)

Can you do:

# vxdisk list Disk_0 ### ie: the disk that shows rootdg imported
# vxdisk list Disk_2 ### or Disk_3/4/5, etc ie: one of the disks showing rootdg deported

Just want to check they're the same rootdg (ie: it's not 2 diff dgs that are both named rootdg)

Also, please confirm the VxVM patch level with

# modinfo | egrep -i "vx|vrts"

#################### EDIT ####################

Just saw your earlier reply indicating you'd created another rootdg (on Disk_0) - if this is a new dg, then the dgid would be different.

If you were on a more recent version, I would say to deport the new rootdg and import it with a new name (or destroy if there's no data on it), and import the original rootdg using the dgid (in vxdisk list <one_of_the_disks>. The problem is that this is 3.5 so you can't deport rootdg!

You could temporarily import the old dg with a new name to check if the data was there, but if you need it to be named rootdg ongoing, you may need to reboot. However, you'd also need to remove/rename the "new" rootdg to avoid conflicts when it tries to import on boot.

Gaurav_S · ‎04-08-2011

thats right ... even I would think of best choice right now is to import the old rootdg with a new name (see help of vxdg) command..

But again, thing we would like to know is clear

-- whether both rootdg's are different, compare the dgids...

-- whether the disks in deported rootdg have config copies or no.. to check this you need to do

# vxdisk list <Disk_2 to Disk_7> |grep -i config

we are interested in these lines:

config priv 000048-000239[000192]: copy=01 offset=000000 disabled
config priv 000256-048207[047952]: copy=01 offset=000192 disabled

how many of those deported disks have an enabled config ...

G

kholcomb6312 · ‎04-08-2011

All enabled but the last...

(root)> vxdisk list Disk_2 | grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled

(root)> vxdisk list Disk_3 | grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled

(root)> vxdisk list Disk_4 | grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled

(root)> vxdisk list Disk_5 | grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled

(root)> vxdisk list Disk_6 | grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled

(root)> vxdisk list Disk_7 | grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 disabled
config   priv 000249-023986[023738]: copy=01 offset=000231 disabled

kholcomb6312 · ‎04-08-2011

Disk_0 and Disk_1 are new LUN's and do not contain any of the original data. All of the original data is on Disk_02 through Disk_07, and I've copied that into the post below. Here is the modinfo:

(root)> modinfo | egrep -i "vx|vrts"
39 102cfd2d 19e84 229   1 vxdmp (VxVM 3.5s DMP Driver)
40 7822e000 116e08 230   1 vxio (VxVM 3.5s_PointPatch1.3 I/O dri)
42 102e7691    80f 231   1 vxspec (VxVM 3.5s control/status driver)

devans3428 · ‎04-08-2011

here you go...notice the privlen of Disk_0 is different. I created a new rootdg with Disk_0 as this was an unused disk thus the only disk with vxdisk list that shows online without the (rootdg) output from vxdisk list.

# modinfo |egrep -i "vx|vrts"
39 102cfd2d 19e84 229   1 vxdmp (VxVM 3.5s DMP Driver)
40 7822e000 116e08 230   1 vxio (VxVM 3.5s_PointPatch1.3 I/O dri)
42 102e7691    80f 231   1 vxspec (VxVM 3.5s control/status driver)

# vxdisk list Disk_2 |grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
# vxdisk list Disk_3 |grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
# vxdisk list Disk_4 |grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
# vxdisk list Disk_5 |grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
# vxdisk list Disk_6 |grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
# vxdisk list Disk_7 |grep -i config
flags:     online ready private autoconfig autoimport
configs:   count=1 len=23969
config   priv 000017-000247[000231]: copy=01 offset=000000 disabled
config   priv 000249-023986[023738]: copy=01 offset=000231 disabled
# vxdisk list Disk_0 |grep -i config
flags:     online ready private autoconfig autoimport imported
configs:   count=1 len=15096
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-015113[014865]: copy=01 offset=000231 enabled

# vxdisk list Disk_0
Device:    Disk_0
devicetag: Disk_0
type:      sliced
hostid:    ncsun1
disk:      name=disk01 id=1302235001.1049.ncsun1
group:     name=rootdg id=1302234996.1025.ncsun1
flags:     online ready private autoconfig autoimport imported
pubpaths: block=/dev/vx/dmp/Disk_0s4 char=/dev/vx/rdmp/Disk_0s4
privpaths: block=/dev/vx/dmp/Disk_0s3 char=/dev/vx/rdmp/Disk_0s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=629094400
private:   slice=3 offset=1 len=20479
update:    time=1302241535 seqno=0.17
headers:   0 248
configs:   count=1 len=15096
logs:      count=1 len=2287
Defined regions:
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-015113[014865]: copy=01 offset=000231 enabled
log      priv 015114-017400[002287]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   2
c10t500601613B241045d7s2        state=enabled
c16t500601603B241045d7s2        state=enabled
# vxdisk list Disk_2
Device:    Disk_2
devicetag: Disk_2
type:      sliced
hostid:    ncsun1
disk:      name= id=1302111549.6037.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
flags:     online ready private autoconfig autoimport
pubpaths: block=/dev/vx/dmp/Disk_2s4 char=/dev/vx/rdmp/Disk_2s4
privpaths: block=/dev/vx/dmp/Disk_2s3 char=/dev/vx/rdmp/Disk_2s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1048494080
private:   slice=3 offset=1 len=32511
update:    time=1302228190 seqno=0.47
headers:   0 248
configs:   count=1 len=23969
logs:      count=1 len=3631
Defined regions:
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
log      priv 023987-027617[003631]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   2
c10t500601613B241045d5s2        state=enabled
c16t500601603B241045d5s2        state=enabled
# vxdisk list Disk_3
Device:    Disk_3
devicetag: Disk_3
type:      sliced
hostid:    ncsun1
disk:      name= id=1302111766.6042.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
flags:     online ready private autoconfig autoimport
pubpaths: block=/dev/vx/dmp/Disk_3s4 char=/dev/vx/rdmp/Disk_3s4
privpaths: block=/dev/vx/dmp/Disk_3s3 char=/dev/vx/rdmp/Disk_3s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1048494080
private:   slice=3 offset=1 len=32511
update:    time=1302228190 seqno=0.47
headers:   0 248
configs:   count=1 len=23969
logs:      count=1 len=3631
Defined regions:
config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
log      priv 023987-027617[003631]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   2
c10t500601613B241045d4s2        state=enabled
c16t500601603B241045d4s2        state=enabled

g_lee · ‎04-08-2011

OK, so we've confirmed the rootdgs are different disk groups, and the config copies are enabled (and it's a hideously old version of 3.5 as well ie: not even patched).

If you want to try to import the old rootdg with a new name to check the data is still there, then

# vxdg -n <newname-dg> import 1072877341.1025.nc1

where the number is the dgid from vxdisk list <disk> output ie:

# vxdisk list Disk_3
Device:    Disk_3
devicetag: Disk_3
type:      sliced
hostid:    ncsun1
disk:      name= id=1302111766.6042.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
flags:     online ready private autoconfig autoimport

If you can reference the volumes by <newname> then you don't need to do anything else.

If you need to reference it by rootdg (ie: you can't update other configs to use a new dg name) then I'd say the best bet is to take a backup of vxprint, vxdisk, etc output (ie: save vxprint -m -g <newdg> output to a file so you can use vxmake to remake the dg, and take note of the offsets, etc used to initialise each disk), then rebuild rootdg using the backed up vxmake file.

Gaurav_S · ‎04-08-2011

yep I am with Grace ... try the above commands. .

I am just wondering if serial split brain comes in picture here .. since we have two rootdg's ... but lets see.

G

devans3428 · ‎04-08-2011

Oh my goodness ....MAN YOU GUYS ROCK!!!!!!!!! everything is there...As you see this is very old stuff so we are just trying to salvage the customer before their departure. So we couldnt spend any money to upgrade. Now once i bring the volumes back online is there any way to reclaim the empty disk disk01 from rootdg to data01 which would remove the rootdg volume leaving only the data01 volume. Basicly i want to move disk01 to data01.

# vxdg -n data01 import 1072877341.1025.nc1
vxvm:vxconfigd: WARNING: Disk Disk_0 names group rootdg, but group ID differs
# vxdisk list
DEVICE       TYPE      DISK         GROUP        STATUS
Disk_0       sliced    disk01       rootdg       online
Disk_1       sliced    -            -            error
Disk_2       sliced    disk05       data01       online
Disk_3       sliced    disk06       data01       online
Disk_4       sliced    disk07       data01       online
Disk_5       sliced    disk08       data01       online
Disk_6       sliced    disk09       data01       online
Disk_7       sliced    disk10       data01       online
c1t0d0s2     sliced    -            -            error
c1t1d0s2     sliced    -            -            error
# vxprint -Aht
Disk group: rootdg

DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
DM NAME         DEVICE       TYPE     PRIVLEN PUBLEN   STATE
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY DATAVOLS SRL
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
V NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO

dg rootdg default default 0 1302234996.1025.ncsun1

dm disk01 Disk_0 sliced 20479 629094400 -

Disk group: data01

DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
DM NAME         DEVICE       TYPE     PRIVLEN PUBLEN   STATE
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY DATAVOLS SRL
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
V NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO

dg data01 default default 0 1072877341.1025.nc1

dm disk05       Disk_2       sliced   32511    1048494080 -
dm disk06       Disk_3       sliced   32511    1048494080 -
dm disk07       Disk_4       sliced   32511    1048494080 -
dm disk08       Disk_5       sliced   32511    1048494080 -
dm disk09       Disk_6       sliced   32511    1048494080 -
dm disk10       Disk_7       sliced   32511    1048494080 -

v NB5.1_Client -            DISABLED ACTIVE   2097152 SELECT    -        fsgen
pl NB5.1_Client-02 NB5.1_Client DISABLED ACTIVE 2097152 CONCAT    -        RW
sd disk06-03    NB5.1_Client-02 disk06 479199232 2097152 0        Disk_3   ENA

v NCFADS_Scripts -          DISABLED ACTIVE   4194304 SELECT    -        fsgen
pl NCFADS_Scripts-02 NCFADS_Scripts DISABLED ACTIVE 4194304 CONCAT -       RW
sd disk07-08    NCFADS_Scripts-02 disk07 10485760 4194304 0       Disk_4   ENA

v NDH_Home     -            DISABLED ACTIVE   146800640 SELECT   -        fsgen
pl NDH_Home-02 NDH_Home     DISABLED ACTIVE   146800640 CONCAT   -        RW
sd disk08-08    NDH_Home-02 disk08   732119040 146800640 0       Disk_5   ENA

v Perl_5.8.3   -            DISABLED ACTIVE   2097152 SELECT    -        fsgen
pl Perl_5.8.3-03 Perl_5.8.3 DISABLED ACTIVE   2097152 CONCAT    -        RW
sd disk07-05    Perl_5.8.3-03 disk07 0        2097152 0         Disk_4   ENA

v Python_2.3.4 -            DISABLED ACTIVE   2097152 SELECT    -        fsgen
pl Python_2.3.4-03 Python_2.3.4 DISABLED ACTIVE 2097152 CONCAT    -        RW
sd disk08-06    Python_2.3.4-03 disk08 721633280 2097152 0        Disk_5   ENA

v Python_2.3.4_Beta - DISABLED ACTIVE 1048576 SELECT - fsgen
pl Python_2.3.4_Beta-03 Python_2.3.4_Beta DISABLED ACTIVE 1048576 CONCAT - RW
sd disk09-08 Python_2.3.4_Beta-03 disk09 939524096 1048576 0 Disk_6 ENA

v SAS_ODBC     -            DISABLED ACTIVE   4194304 SELECT    -        fsgen
pl SAS_ODBC-02 SAS_ODBC     DISABLED ACTIVE   4194304 CONCAT    -        RW
sd disk05-02    SAS_ODBC-02 disk05   629145600 4194304 0         Disk_2   ENA

v SAS82        -            DISABLED ACTIVE   4194304 SELECT    -        fsgen
pl SAS82-02     SAS82        DISABLED ACTIVE   4194304 CONCAT    -        RW
sd disk06-04    SAS82-02     disk06   481296384 4194304 0         Disk_3   ENA

v SybData2     -            DISABLED ACTIVE   12582912 SELECT    -        fsgen
pl SybData2-02 SybData2     DISABLED ACTIVE   12582912 CONCAT    -        RW
sd disk10-09    SybData2-02 disk10   987758592 12582912 0        Disk_7   ENA

v Sybase12V5   -            DISABLED ACTIVE   8388608 SELECT    -        fsgen
pl Sybase12V5-03 Sybase12V5 DISABLED ACTIVE   8388608 CONCAT    -        RW
sd disk07-10    Sybase12V5-03 disk07 2097152 8388608 0         Disk_4   ENA

v archive      -            DISABLED ACTIVE   8388608 SELECT    -        gen
pl archive-03   archive      DISABLED ACTIVE   8388608 CONCAT    -        RW
sd disk08-09    archive-03   disk08   723730432 8388608 0         Disk_5   ENA

v home         -            DISABLED ACTIVE   8388608 SELECT    -        gen
pl home-02      home         DISABLED ACTIVE   8388608 CONCAT    -        RW
sd disk08-10    home-02      disk08   704856064 8388608 0         Disk_5   ENA

v java         -            DISABLED ACTIVE   3145728 SELECT    -        fsgen
pl java-02      java         DISABLED ACTIVE   3145728 CONCAT    -        RW
sd disk09-10    java-02      disk09   944766976 3145728 0         Disk_6   ENA

v mfserver     -            DISABLED ACTIVE   2097152 SELECT    -        gen
pl mfserver-02 mfserver     DISABLED ACTIVE   2097152 CONCAT    -        RW
sd disk07-11    mfserver-02 disk07   14680064 2097152 0         Disk_4   ENA

v ncdss        -            DISABLED ACTIVE   629145600 SELECT   -        gen
pl ncdss-02     ncdss        DISABLED ACTIVE   629145600 CONCAT   -        RW
sd disk05-01    ncdss-02     disk05   0        629145600 0        Disk_2   ENA

v ncusers      -            DISABLED ACTIVE   314572800 SELECT   -        fsgen
pl ncusers-02   ncusers      DISABLED ACTIVE   314572800 CONCAT   -        RW
sd disk06-01    ncusers-02   disk06   0        314572800 0        Disk_3   ENA

v nharch       -            DISABLED ACTIVE   4194304 SELECT    -        gen
pl nharch-03    nharch       DISABLED ACTIVE   4194304 CONCAT    -        RW
sd disk09-09    nharch-03    disk09   940572672 4194304 0         Disk_6   ENA

v sasprod      -            DISABLED ACTIVE   125829120 SELECT   -        gen
pl sasprod-02   sasprod      DISABLED ACTIVE   125829120 CONCAT   -        RW
sd disk06-02    sasprod-02   disk06   314572800 125829120 0       Disk_3   ENA

v sastemp      -            DISABLED ACTIVE   41943040 SELECT    -        gen
pl sastemp-02   sastemp      DISABLED ACTIVE   41943040 CONCAT    -        RW
sd disk06-14    sastemp-02   disk06   490733568 41943040 0        Disk_3   ENA

v sas8         -            DISABLED ACTIVE   2097152 SELECT    -        gen
pl sas8-02      sas8         DISABLED ACTIVE   2097152 CONCAT    -        RW
sd disk06-16    sas8-02      disk06   545259520 2097152 0         Disk_3   ENA

v sas81        -            DISABLED ACTIVE   2097152 SELECT    -        gen
pl sas81-02     sas81        DISABLED ACTIVE   2097152 CONCAT    -        RW
sd disk06-08    sas81-02     disk06   532676608 2097152 0         Disk_3   ENA

v sybdebug     -            DISABLED ACTIVE   5242880 SELECT    -        fsgen
pl sybdebug-02 sybdebug     DISABLED ACTIVE   5242880 CONCAT    -        RW
sd disk06-17    sybdebug-02 disk06   547356672 5242880 0         Disk_3   ENA

v sybtest      -            DISABLED ACTIVE   12582912 SELECT    -        fsgen
pl sybtest-02   sybtest      DISABLED ACTIVE   12582912 CONCAT    -        RW
sd disk06-09    sybtest-02   disk06   466616320 12582912 0        Disk_3   ENA

v vol200       -            DISABLED ACTIVE   67108864 SELECT    -        gen
pl vol200-02    vol200       DISABLED ACTIVE   67108864 CONCAT    -        RW
sd disk05-03    vol200-02    disk05   775946240 67108864 0        Disk_2   ENA

v vol201       -            DISABLED ACTIVE   67108864 SELECT    -        gen
pl vol201-02    vol201       DISABLED ACTIVE   67108864 CONCAT    -        RW
sd disk05-04    vol201-02    disk05   843055104 67108864 0        Disk_2   ENA

v vol202       -            DISABLED ACTIVE   8388608 SELECT    -        gen
pl vol202-02    vol202       DISABLED ACTIVE   8388608 CONCAT    -        RW
sd disk06-18    vol202-02    disk06   552599552 8388608 0         Disk_3   ENA

v vol203       -            DISABLED ACTIVE   8388608 SELECT    -        gen
pl vol203-02    vol203       DISABLED ACTIVE   8388608 CONCAT    -        RW
sd disk06-19    vol203-02    disk06   560988160 8388608 0         Disk_3   ENA

v vol400       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol400-02    vol400       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk07-06    vol400-02    disk07   27262976 134217728 0        Disk_4   ENA

v vol401       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol401-02    vol401       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk07-07    vol401-02    disk07   161480704 134217728 0       Disk_4   ENA

v vol402       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol402-02    vol402       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk06-20    vol402-02    disk06   569376768 134217728 0       Disk_3   ENA

v vol403       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol403-02    vol403       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk06-21    vol403-02    disk06   703594496 134217728 0       Disk_3   ENA

v vol404       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol404-02    vol404       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk05-05    vol404-02    disk05   910163968 134217728 0       Disk_2   ENA

v vol405       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol405-02    vol405       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk06-22    vol405-02    disk06   837812224 134217728 0       Disk_3   ENA

v vol406       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol406-02    vol406       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk07-03    vol406-02    disk07   597688320 134217728 0       Disk_4   ENA

v vol407       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol407-02    vol407       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk08-04    vol407-02    disk08   301989888 134217728 0       Disk_5   ENA

v vol408       -            DISABLED ACTIVE   33554432 SELECT    -        gen
pl vol408-02    vol408       DISABLED ACTIVE   33554432 CONCAT    -        RW
sd disk06-23    vol408-02    disk06   972029952 33554432 0        Disk_3   ENA

v vol409       -            DISABLED ACTIVE   33554432 SELECT    -        gen
pl vol409-02    vol409       DISABLED ACTIVE   33554432 CONCAT    -        RW
sd disk06-24    vol409-02    disk06   1005584384 33554432 0       Disk_3   ENA

v vol410       -            DISABLED ACTIVE   33554432 SELECT    -        gen
pl vol410-02    vol410       DISABLED ACTIVE   33554432 CONCAT    -        RW
sd disk07-09    vol410-02    disk07   429916160 33554432 0        Disk_4   ENA

v vol411       -            DISABLED ACTIVE   33554432 SELECT    -        gen
pl vol411-02    vol411       DISABLED ACTIVE   33554432 CONCAT    -        RW
sd disk08-02    vol411-02    disk08   134217728 33554432 0        Disk_5   ENA

v vol412       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol412-02    vol412       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk07-02    vol412-02    disk07   463470592 134217728 0       Disk_4   ENA

v vol413       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol413-02    vol413       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk08-03    vol413-02    disk08   167772160 134217728 0       Disk_5   ENA

v vol414       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol414-02    vol414       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-05    vol414-02    disk09   536870912 134217728 0       Disk_6   ENA

v vol415       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol415-02    vol415       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-05    vol415-02    disk10   536870912 134217728 0       Disk_7   ENA

v vol416       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol416-02    vol416       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-04    vol416-02    disk09   402653184 134217728 0       Disk_6   ENA

v vol417       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol417-02    vol417       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-04    vol417-02    disk10   402653184 134217728 0       Disk_7   ENA

v vol418       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol418-02    vol418       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk07-01    vol418-02    disk07   295698432 134217728 0       Disk_4   ENA

v vol419       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol419-02    vol419       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk08-01    vol419-02    disk08   0        134217728 0        Disk_5   ENA

v vol420       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol420-02    vol420       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-01    vol420-02    disk10   0        134217728 0        Disk_7   ENA

v vol421       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol421-02    vol421       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-01    vol421-02    disk09   0        134217728 0        Disk_6   ENA

v vol422       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol422-02    vol422       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-02    vol422-02    disk10   134217728 134217728 0       Disk_7   ENA

v vol423       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol423-02    vol423       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-02    vol423-02    disk09   134217728 134217728 0       Disk_6   ENA

v vol424       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol424-02    vol424       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-03    vol424-02    disk10   268435456 134217728 0       Disk_7   ENA

v vol425       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol425-02    vol425       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-03    vol425-02    disk09   268435456 134217728 0       Disk_6   ENA

v vol426       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol426-02    vol426       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-06    vol426-02    disk10   671088640 134217728 0       Disk_7   ENA

v vol427       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol427-02    vol427       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-06    vol427-02    disk09   671088640 134217728 0       Disk_6   ENA

v vol428       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol428-02    vol428       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk10-07    vol428-02    disk10   805306368 134217728 0       Disk_7   ENA

v vol429       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol429-02    vol429       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk09-07    vol429-02    disk09   805306368 134217728 0       Disk_6   ENA

v vol430       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol430-02    vol430       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk08-07    vol430-02    disk08   570638336 134217728 0       Disk_5   ENA

v vol431       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol431-02    vol431       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk07-04    vol431-02    disk07   731906048 134217728 0       Disk_4   ENA

v vol432       -            DISABLED ACTIVE   134217728 SELECT   -        gen
pl vol432-02    vol432       DISABLED ACTIVE   134217728 CONCAT   -        RW
sd disk08-05    vol432-02    disk08   436207616 134217728 0       Disk_5   ENA

v vol500       -            DISABLED ACTIVE   20971520 SELECT    -        fsgen
pl vol500-02    vol500       DISABLED ACTIVE   20971520 CONCAT    -        RW
sd disk06-05    vol500-02    disk06   457179136 9437184 0         Disk_3   ENA
sd disk06-06    vol500-02    disk06   486539264 2179072 9437184   Disk_3   ENA
sd disk06-07    vol500-02    disk06   1039138816 9355264 11616256 Disk_3   ENA

v vol600       -            DISABLED ACTIVE   204800   SELECT    -        gen
pl vol600-02    vol600       DISABLED ACTIVE   212992   CONCAT    -        RW
sd disk08-11    vol600-02    disk08   570425344 212992 0         Disk_5   ENA

v vol601       -            DISABLED ACTIVE   245760   SELECT    -        gen
pl vol601-02    vol601       DISABLED ACTIVE   245760   CONCAT    -        RW
sd disk09-11    vol601-02    disk09   947912704 245760 0         Disk_6   ENA

v vol602       -            DISABLED ACTIVE   16777216 SELECT    -        gen
pl vol602-02    vol602       DISABLED ACTIVE   16777216 CONCAT    -        RW
sd disk07-12    vol602-02    disk07   866123776 16777216 0        Disk_4   ENA

v vol603       -            DISABLED ACTIVE   4194304 SELECT    -        gen
pl vol603-02    vol603       DISABLED ACTIVE   4194304 CONCAT    -        RW
sd disk07-13    vol603-02    disk07   16777216 4194304 0         Disk_4   ENA

v vol604       -            DISABLED ACTIVE   67108864 SELECT    -        gen
pl vol604-02    vol604       DISABLED ACTIVE   67108864 CONCAT    -        RW
sd disk08-12    vol604-02    disk08   878919680 67108864 0        Disk_5   ENA

v vol605       -            DISABLED ACTIVE   10485760 SELECT    -        gen
pl vol605-03    vol605       DISABLED ACTIVE   10485760 CONCAT    -        RW
sd disk10-10    vol605-03    disk10   977272832 10485760 0        Disk_7   ENA

v vol606       -            DISABLED ACTIVE   33554432 SELECT    -        gen
pl vol606-03    vol606       DISABLED ACTIVE   33554432 CONCAT    -        RW
sd disk10-08    vol606-03    disk10   939524096 33554432 0        Disk_7   ENA

v vol607       -            DISABLED ACTIVE   4194304 SELECT    -        gen
pl vol607-02    vol607       DISABLED ACTIVE   4194304 CONCAT    -        RW
sd disk07-14    vol607-02    disk07   20971520 4194304 0         Disk_4   ENA

v vol608       -            DISABLED ACTIVE   16777216 SELECT    -        gen
pl vol608-02    vol608       DISABLED ACTIVE   16777216 CONCAT    -        RW
sd disk08-13    vol608-02    disk08   946028544 16777216 0        Disk_5   ENA

Gaurav_S · ‎04-08-2011

Unfortunately NO, you can not move a disk from one diskgroup to other without destroying the private region & re-adding it to new DG ... & since you have root on it , it is further more difficult ..

by the way , whats inside rootdg ? can you paste contents of

# df -k

# vxprint -qthg rootdg

G

kholcomb6312 · ‎04-08-2011

(root)> df -k
Filesystem            kbytes    used   avail capacity Mounted on
/dev/md/dsk/d0       17137652 11610907 5355369    69%    /
/proc                      0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
fd                         0       0       0     0%    /dev/fd
swap                 74235312      40 74235272     1%    /var/run
swap                 74236080     808 74235272     1%    /tmp

(root)> vxprint -qthg rootdg
dg rootdg default default 0 1302234996.1025.ncsun1

dm disk01 Disk_0 sliced 20479 629094400 -

devans3428 · ‎04-08-2011

Kholcomb6132 and I are working the same server and he replied to your df. Further information is that the system root is not under veritas. So rootdg is empty with no data only a blank disk. So with that in mind can i just blow away the group rootdg and reclaim the disk for disk group data01. As you can see there are no volumes associated with rootdg only the one disk

# vxprint -g rootdg -ht
DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
DM NAME         DEVICE       TYPE     PRIVLEN PUBLEN   STATE
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY DATAVOLS SRL
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
V NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO

dg rootdg default default 0 1302234996.1025.ncsun1

dm disk01 Disk_0 sliced 20479 629094400 -

So in summary i want to delete the group rootdg(no relation to the system / directory) and claim the Disk_0 to data01 diskgroup. Is that possible?

Gaurav_S · ‎04-08-2011

3.5, rootdg is a must, rootdg should be there otherwise vxvm will go down ... now with running rootdg on a blank disk, you can't deport that diskgroup. I hope you understand...

Now I am really not sure what will happen if you destroy rootdg, then deport data01 dg & reimport it as rootdg .. I am concerned at first step, if you destroy rootdg, not sure your vxconfigd will be up with that step .. even not sure if destroy operation on rootdg will be allowed ... honestly I never tried that..

G

devans3428 · ‎04-08-2011

Ok here is an idea...If i add a smaller disk to rootdg and then remove the current disk from rootdg leaving the smaller disk in rootdg, think that will work? If not we will just use rootdg as it is and put a filesystem on that dg. As you see there are no volumes in rootdg on the one disk. Also before i got everything working i placed vxconfigd in disable mode "vxconfigd -k -m disable". I then ran vxdctl enable and know if i do type "vxdctl mode" it shows it is enable. Is it ok now to run stop vxconfigd and restart as normal "vxconfigd". All the volumes are now enabled and fsck'd and filesystems are mounted. If not we will just use rootdg as it is and create a volume and filesystem on that dg.

# vxprint -g data01 -v
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0 PUTIL0
v NB5.1_Client fsgen        ENABLED 2097152 -        ACTIVE   -       -
v NCFADS_Scripts fsgen      ENABLED 4194304 -        ACTIVE   -       -
v NDH_Home     fsgen        ENABLED 146800640 -       ACTIVE   -       -
v Perl_5.8.3   fsgen        ENABLED 2097152 -        ACTIVE   -       -
v Python_2.3.4 fsgen        ENABLED 2097152 -        ACTIVE   -       -
v Python_2.3.4_Beta fsgen   ENABLED 1048576 -        ACTIVE   -       -
v SAS_ODBC     fsgen        ENABLED 4194304 -        ACTIVE   -       -
v SAS82        fsgen        ENABLED 4194304 -        ACTIVE   -       -
v SybData2     fsgen        ENABLED 12582912 -        ACTIVE   -       -
v Sybase12V5   fsgen        ENABLED 8388608 -        ACTIVE   -       -
v archive      gen          ENABLED 8388608 -        ACTIVE   -       -
v home         gen          ENABLED 8388608 -        ACTIVE   -       -
v java         fsgen        ENABLED 3145728 -        ACTIVE   -       -
v mfserver     gen          ENABLED 2097152 -        ACTIVE   -       -
v ncdss        gen          ENABLED 629145600 -       ACTIVE   -       -
v ncusers      fsgen        ENABLED 314572800 -       ACTIVE   -       -
v nharch       gen          ENABLED 4194304 -        ACTIVE   -       -
v sasprod      gen          ENABLED 125829120 -       ACTIVE   -       -
v sastemp      gen          ENABLED 41943040 -        ACTIVE   -       -
v sas8         gen          ENABLED 2097152 -        ACTIVE   -       -
v sas81        gen          ENABLED 2097152 -        ACTIVE   -       -
v sybdebug     fsgen        ENABLED 5242880 -        ACTIVE   -       -
v sybtest      fsgen        ENABLED 12582912 -        ACTIVE   -       -

# vxdisk list
DEVICE       TYPE      DISK         GROUP        STATUS
Disk_0       sliced    disk01       rootdg       online
Disk_1       sliced    -            -            error
Disk_2       sliced    disk05       data01       online
Disk_3       sliced    disk06       data01       online
Disk_4       sliced    disk07       data01       online
Disk_5       sliced    disk08       data01       online
Disk_6       sliced    disk09       data01       online
Disk_7       sliced    disk10       data01       online

Any final thoughts here?

Gaurav_S · ‎04-08-2011

That should be fine, you can add a small disk to rootdg if you have & get your needed disk removed ...

Once your disk is place under rootdg, I don't think there should be any problem in restarting vxconfigd as normal ..

You need to remember one more thing, if you are mounting all the volumes via vfstab. then do modify the vfstab to reflect data01 instead of rootdg else server will thorow errors while booting...

G

devans3428 · ‎04-08-2011

Ok ...thanks for getting me back up and running. Everything is fine.

VOX

Dynamic multipath using EMC storage Veritas 3.5