cancel
Showing results for 
Search instead for 
Did you mean: 

Dynamic multipath using EMC storage Veritas 3.5

devans3428
Level 4

I am trying to setup multipathing with EMC Clariion.   The problem is that vxdisk list fabric_0 only shows one path.   The EMC array is in auto-trespass mode.   This is solaris 8 and format shows two paths.

# vxdisk list fabric_2
Device:    fabric_2
devicetag: fabric_2
type:      sliced
hostid:    ncsun1
disk:      name=disk05 id=1302111549.6037.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
info:      privoffset=1
flags:     online ready private autoconfig autoimport imported
pubpaths:  block=/dev/vx/dmp/fabric_2s4 char=/dev/vx/rdmp/fabric_2s4
privpaths: block=/dev/vx/dmp/fabric_2s3 char=/dev/vx/rdmp/fabric_2s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1048494080
private:   slice=3 offset=1 len=32511
update:    time=1302111558 seqno=0.5
headers:   0 248
configs:   count=1 len=23969
logs:      count=1 len=3631
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
 log      priv 023987-027617[003631]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c10t500601613B241045d5s2        state=enabled
 

formt

8. c10t500601613B241045d0 <DGC-RAID5-0428 cyl 63998 alt 2 hd 256 sec 64>
          /ssm@0,0/pci@19,700000/SUNW,qlc@2/fp@0,0/ssd@w500601613b241045,0
 

16. c16t500601603B241045d0 <DGC-RAID5-0428 cyl 63998 alt 2 hd 256 sec 64>
          /ssm@0,0/pci@18,700000/SUNW,qlc@1/fp@0,0/ssd@w500601603b241045,0
 

vxdisk -o alldgs list show both paths.  Two things here it should only show one of the paths and also the second path it shows with diskgroup in ( ).  Another issue is why the disk dont show up as EMC_0 or similiar.

*****The server has T3 connect and EMC which we are migrating from T3 to EMC.    The EMC is the fabric naming convention.

# vxdisk -o alldgs list
DEVICE       TYPE      DISK         GROUP        STATUS
T30_0        sliced    disk01       rootdg       online
T30_1        sliced    disk02       rootdg       online
T31_0        sliced    disk03       rootdg       online
T31_1        sliced    disk04       rootdg       online
T32_0        sliced    rootdg00     rootdg       online
T32_1        sliced    rootdg01     rootdg       online
c1t0d0s2     sliced    -            -            error
c1t1d0s2     sliced    -            -            error
fabric_0     sliced    -            -            error
fabric_1     sliced    -            -            error
fabric_2     sliced    disk05       rootdg       online
fabric_3     sliced    disk06       rootdg       online
fabric_4     sliced    disk07       rootdg       online
fabric_5     sliced    disk08       rootdg       online
fabric_6     sliced    disk09       rootdg       online
fabric_7     sliced    disk10       rootdg       online
fabric_8     sliced    -            -            error
fabric_9     sliced    -            -            error
fabric_10    sliced    -           (rootdg)      online
fabric_11    sliced    -           (rootdg)      online
fabric_12    sliced    -           (rootdg)      online
fabric_13    sliced    -           (rootdg)      online
fabric_14    sliced    -           (rootdg)      online
fabric_15    sliced    -           (rootdg)      online

Here is the ASL...There is no APM prior to version Veritas 4.0.

vxddladm listsupport

snippet for brevity.....

libvxDGCclariion.so              A/P          DGC          CLARiiON

The c10 and c16 are the paths for the EMC

# vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c1              OTHER_DISKS     ENABLED      OTHER_DISKS
c10             OTHER_DISKS     ENABLED      OTHER_DISKS
c16             OTHER_DISKS     ENABLED      OTHER_DISKS
 

# vxdmpadm getsubpaths ctlr=c10
NAME         STATE         PATH-TYPE  DMPNODENAME  ENCLR-TYPE   ENCLR-NAME
======================================================================
c10t500601613B241045d7s2 ENABLED        -        fabric_0     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d6s2 ENABLED        -        fabric_1     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d5s2 ENABLED        -        fabric_2     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d4s2 ENABLED        -        fabric_3     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d3s2 ENABLED        -        fabric_4     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d2s2 ENABLED        -        fabric_5     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d1s2 ENABLED        -        fabric_6     OTHER_DISKS  OTHER_DISKS
c10t500601613B241045d0s2 ENABLED        -        fabric_7     OTHER_DISKS  OTHER_DISKS
# vxdmpadm getsubpaths ctlr=c16
NAME         STATE         PATH-TYPE  DMPNODENAME  ENCLR-TYPE   ENCLR-NAME
======================================================================
c16t500601603B241045d7s2 ENABLED        -        fabric_8     OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d6s2 ENABLED        -        fabric_9     OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d5s2 ENABLED        -        fabric_10    OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d4s2 ENABLED        -        fabric_11    OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d3s2 ENABLED        -        fabric_12    OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d2s2 ENABLED        -        fabric_13    OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d1s2 ENABLED        -        fabric_14    OTHER_DISKS  OTHER_DISKS
c16t500601603B241045d0s2 ENABLED        -        fabric_15    OTHER_DISKS  OTHER_DISKS
 

Thanks for any help

1 ACCEPTED SOLUTION

Accepted Solutions

devans3428
Level 4

Ok here is an idea...If i add a smaller disk to rootdg and then remove the current disk from rootdg leaving the smaller disk in rootdg, think that will work?   If not we will just use rootdg as it is and put a filesystem on that dg. As you see there are no volumes in rootdg on the one disk.  Also before i got everything working i placed vxconfigd in disable mode "vxconfigd -k -m disable".  I then ran vxdctl enable and know if i do type "vxdctl mode" it shows it is enable.   Is it ok now to run stop vxconfigd and restart as normal "vxconfigd".   All the volumes are now enabled and fsck'd and filesystems are mounted.   If not we will just use rootdg as it is and create a volume and filesystem on that dg.  

# vxprint -g data01 -v
TY NAME         ASSOC        KSTATE   LENGTH   PLOFFS   STATE    TUTIL0  PUTIL0
v  NB5.1_Client fsgen        ENABLED  2097152  -        ACTIVE   -       -
v  NCFADS_Scripts fsgen      ENABLED  4194304  -        ACTIVE   -       -
v  NDH_Home     fsgen        ENABLED  146800640 -       ACTIVE   -       -
v  Perl_5.8.3   fsgen        ENABLED  2097152  -        ACTIVE   -       -
v  Python_2.3.4 fsgen        ENABLED  2097152  -        ACTIVE   -       -
v  Python_2.3.4_Beta fsgen   ENABLED  1048576  -        ACTIVE   -       -
v  SAS_ODBC     fsgen        ENABLED  4194304  -        ACTIVE   -       -
v  SAS82        fsgen        ENABLED  4194304  -        ACTIVE   -       -
v  SybData2     fsgen        ENABLED  12582912 -        ACTIVE   -       -
v  Sybase12V5   fsgen        ENABLED  8388608  -        ACTIVE   -       -
v  archive      gen          ENABLED  8388608  -        ACTIVE   -       -
v  home         gen          ENABLED  8388608  -        ACTIVE   -       -
v  java         fsgen        ENABLED  3145728  -        ACTIVE   -       -
v  mfserver     gen          ENABLED  2097152  -        ACTIVE   -       -
v  ncdss        gen          ENABLED  629145600 -       ACTIVE   -       -
v  ncusers      fsgen        ENABLED  314572800 -       ACTIVE   -       -
v  nharch       gen          ENABLED  4194304  -        ACTIVE   -       -
v  sasprod      gen          ENABLED  125829120 -       ACTIVE   -       -
v  sastemp      gen          ENABLED  41943040 -        ACTIVE   -       -
v  sas8         gen          ENABLED  2097152  -        ACTIVE   -       -
v  sas81        gen          ENABLED  2097152  -        ACTIVE   -       -
v  sybdebug     fsgen        ENABLED  5242880  -        ACTIVE   -       -
v  sybtest      fsgen        ENABLED  12582912 -        ACTIVE   -       -
 

# vxdisk list
DEVICE       TYPE      DISK         GROUP        STATUS
Disk_0       sliced    disk01       rootdg       online
Disk_1       sliced    -            -            error
Disk_2       sliced    disk05       data01       online
Disk_3       sliced    disk06       data01       online
Disk_4       sliced    disk07       data01       online
Disk_5       sliced    disk08       data01       online
Disk_6       sliced    disk09       data01       online
Disk_7       sliced    disk10       data01       online
 

Any final thoughts here?

View solution in original post

34 REPLIES 34

devans3428
Level 4

OS can see both paths seems something is with the ASL....Oh and let me add I am not using navisphere or powerpath....

# luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
  Node WWN:50020f200000bd4f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F200000C2DC44EF5AF7000B560Dd0s2
  Node WWN:50020f200000bd4f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F200000C2DC44EF593900005A36d0s2
  Node WWN:50020f200001062f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000104AE3EB2EF4A000C3565d0s2
  Node WWN:50020f200001062f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000104AE3EB2EF5A00058465d0s2
  Node WWN:50020f2000010c33  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000107DC3EB2F021000BEB2Dd0s2
  Node WWN:50020f2000010c33  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000107DC3EB2F0310003CB97d0s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d0s2
    Logical Path:/dev/rdsk/c16t500601603B241045d0s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d1s2
    Logical Path:/dev/rdsk/c16t500601603B241045d1s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d2s2
    Logical Path:/dev/rdsk/c16t500601603B241045d2s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d3s2
    Logical Path:/dev/rdsk/c16t500601603B241045d3s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d4s2
    Logical Path:/dev/rdsk/c16t500601603B241045d4s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d5s2
    Logical Path:/dev/rdsk/c16t500601603B241045d5s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d6s2
    Logical Path:/dev/rdsk/c16t500601603B241045d6s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d7s2
    Logical Path:/dev/rdsk/c16t500601603B241045d7s2
#

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hello,

So its version 4.0 for vxvm, can you confirm the exact version ?

# pkginfo -l VRTSvxvm

Also, I see that you have ASL however somehow I believe either ASL is in excluded state or else its not claiming the devices...

Can you  paste some outputs:

# vxdmpadm listexclude all

# ls -l /etc/vx/*exclude*

if you find any exclude file, paste the contents of them...

Also, can you attach output of:

# /etc/vx/diag.d/vxasldebug    (if path is wrong, check in other directories for same command, not sure of exact path in 4.0)

Paths not being shown & vxdisk list showing as "fabric" both are relating to ASL issue only .. Tell me this also:

1. is this a new installation ? was it working before ?

2. whether ASL was installed manually or it came pre-installed ?

 

 

Gaurav

devans3428
Level 4

Hi Gaurav,

Sorry i thought i mention the version in my previous but i see i didnt.   Its 3.5 see below:

# pkginfo -l VRTSvxvm
   PKGINST:  VRTSvxvm
      NAME:  VERITAS Volume Manager, Binaries
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  3.5,REV=06.21.2002.23.14
   BASEDIR:  /
    VENDOR:  VERITAS Software
      DESC:  Virtual Disk Subsystem
    PSTAMP:  VERITAS-3.5s_PointPatch1.3:26-July-2002
  INSTDATE:  Dec 30 2003 20:16
   HOTLINE:  800-342-0652
     EMAIL:  support@veritas.com
    STATUS:  completely installed
     FILES:      600 installed pathnames
                  21 shared pathnames
                   9 linked files
                  76 directories
                 337 executables
              159005 blocks used (approx)

 

# vxdmpadm listexclude all
Devices excluded from VxVM:
--------------------------
Paths : None

Controllers : None

VID:PID : None

Devices excluded from multipathing by vxdmp:
-------------------------------------------
Paths : None

VID:PID : None

Pathgroups : None

# ls -l /etc/vx/*exclude*
-rw-r--r--   1 root     other         59 Dec 31  2003 /etc/vx/vxdmp.exclude
-rw-r--r--   1 root     other         59 Dec 31  2003 /etc/vx/vxvm.exclude
# cat /etc/vx/vxdmp.exclude
exclude_all 0
paths
#
controllers
#
product
#
pathgroups
#
# cat /etc/vx/vxvm.exclude
exclude_all 0
paths
#
controllers
#
product
#
pathgroups
 

Output for asldebug is attached in gzip format

This is a new installation of using EMC storage.  Before the storage was using T3 SAN.   The ASL library was installed via pkgadd just two days ago.  The T3 are using mpxio for multipathing.  

Here is a T3 disk its using mpxio as you can see by the long naming scheme...not concerned with the T3:

# vxdisk list T30_0
Device:    T30_0
devicetag: T30_0
type:      sliced
hostid:    ncsun1
disk:      name=disk01 id=1072877353.1041.nc1
group:     name=rootdg id=1072877341.1025.nc1
flags:     online ready private autoconfig autoimport imported
pubpaths:  block=/dev/vx/dmp/T30_0s4 char=/dev/vx/rdmp/T30_0s4
privpaths: block=/dev/vx/dmp/T30_0s3 char=/dev/vx/rdmp/T30_0s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1000595456
private:   slice=3 offset=1 len=57343
update:    time=1302107429 seqno=0.343
headers:   0 248
configs:   count=1 len=42303
logs:      count=1 len=6409
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-042320[042072]: copy=01 offset=000231 enabled
 log      priv 042321-048729[006409]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c7t60020F20000104AE3EB2EF5A00058465d0s2 state=enabled   type=primary
 

 

Here is EMC:

# vxdisk list fabric_2
Device:    fabric_2
devicetag: fabric_2
type:      sliced
hostid:    ncsun1
disk:      name=disk05 id=1302111549.6037.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
info:      privoffset=1
flags:     online ready private autoconfig autoimport imported
pubpaths:  block=/dev/vx/dmp/fabric_2s4 char=/dev/vx/rdmp/fabric_2s4
privpaths: block=/dev/vx/dmp/fabric_2s3 char=/dev/vx/rdmp/fabric_2s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1048494080
private:   slice=3 offset=1 len=32511
update:    time=1302111558 seqno=0.5
headers:   0 248
configs:   count=1 len=23969
logs:      count=1 len=3631
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
 log      priv 023987-027617[003631]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c10t500601613B241045d5s2        state=enabled
 

 


 

Gaurav_S
Moderator
Moderator
   VIP    Certified

Ok, first surprise to me that its a new installation & you have installed 3.5 which is End of Life Ages ago .. Even 4.0  & in last OCt even 4.1 was end of life .... so any particular reason you are using such an old version ? 

I would very strongly recommend you to install latest version (5.1SP1 preferrable) .. I am saying this because even if your 3.5 setup gets to work, Symantec won't support it on any case unless your organization have support exception with them...

Ok coming to original problem .. my suspection was right ... you have ASL installed however it doesn't claim the Luns..  from the attached output:

 

checking asl library outputs
----------------------------------------------------------

**********************************************************
                   libvxDGCclariion.so                                     
**********************************************************

    libvxDGCclariion.so:vendor_info()
    VID                                    : DGC
    PID                                    : CLARiiON
    ATYPE                                  : A/P
    ANAME                                  : EMC_CLARiiON

    libvxDGCclariion.key()
    ASL Name                               : libvxDGCclariion.so
    Feature Needed                         : 95
    VxVM Version Needed                    : 32

    Device                                 : /dev/rdsk/c10t500601613B241045d0s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0C0000D4FCCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d1s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 100000DC30CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d2s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 140000E59FCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d3s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 160000E5A4CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d4s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0E0000D843CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d5s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 120000E16ECL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d6s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 5A00004614CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d7s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 6000004CC9CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d0s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0C0000D4FCCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d1s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 100000DC30CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d2s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 140000E59FCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d3s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 160000E5A4CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d4s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0E0000D843CL
    libvxDGCclariion.so:claim_device()     : ERROR

 

you can see all disks in error state ... so now questions are:

-- are you sure you installed correct version of ASL which is supported by 3.5 ?

-- did you reboot the server after ASL installation ? I doubt if 3.5 will accept ASL changes online ?

--again, if you have any latest version of vxvm, it will have clariion ASL inbuilt & will pickup your devices automatically. ...

Think over it ..

 

G

devans3428
Level 4

I may have mislead you...this is not a new install ...the only new thing is adding the EMC clariion tray to the server.   We are not able to upgrade veritas at this time so trying to make this work as is with the new addition of the EMC tray.

I installed from here:

-- are you sure you installed correct version of ASL which is supported by 3.5 ?

http://www.symantec.com/business/support/index?page=content&id=TECH22609

Seems to be the correct one.... 3.5 solaris

-- did you reboot the server after ASL installation ? I doubt if 3.5 will accept ASL changes online ?

I have rebooted a couple of times along with doing reboot reconfigure

--again, if you have any latest version of vxvm, it will have clariion ASL inbuilt & will pickup your devices automatically. ...

Not able to upgrade because not worth the cost as customer is going away in a year so really trying to get this to work on 3.5

Thanks for the quick response....

devans3428
Level 4

Well we decided since we mirrored the T3 plex with EMC plex under one volume and one disk group we would go ahead and remove the T3 plex and leave the EMC plex in place.  This would leave just the one EMC plex and no T3 volumes exist now.  Upon reboot vxconfigd does not start because of message below

# vxconfigd
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_2, fabric_10
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_3, fabric_11
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_4, fabric_12
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_5, fabric_13
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_6, fabric_14
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_7, fabric_15
vxvm:vxconfigd: ERROR: enable failed: Error in disk group configuration copies
        Unexpected kernel error in configuration update; transactions are disabled.

 

Any idea how to resolve this one?   I think it all is an issue with the DGC library as you have stated.

Thanks
 

Gaurav_S
Moderator
Moderator
   VIP    Certified

well true ..

try out these steps:

# rm /dev/vx/dmp/*

# rm /dev/vx/rdmp/*

# rm /dev/rdsk/*

# rm /dev/rdsk/*

# devfsadm -Cv

# /sbin/vxconfigd -k -m disable

check if vxconfigd started

# vxdctl mode   (see if it returns disabled)

Try to enable it..

# vxdctl enable

 

Also, can u paste output of:

pkginfo -l CLR-ASL

 

Gaurav

devans3428
Level 4

Did all the steps above and the results are below....

# vxdctl enable
vxvm:vxconfigd: ERROR: enable failed: Error in disk group configuration copies
        No valid disk found containing disk group; transactions are disabled.
vxvm:vxdctl: ERROR: enable failed: Error in disk group configuration copies
# pkginfo -l CLR-ASL
   PKGINST:  CLR-ASL
      NAME:  EMC CLARIION Array Support Library
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  1.04,REV=07.25.2002.17.36
   BASEDIR:  /etc/vx
    VENDOR:  EMC Corp
      DESC:  EMC CLARIION Array Support Library
    PSTAMP:  VERSION-1.04:14-February-2002
  INSTDATE:  Apr 07 2011 18:00
   HOTLINE:  800-782-4362
    STATUS:  completely installed
     FILES:        4 installed pathnames
                   2 shared pathnames
                   2 directories
                   2 executables
                  68 blocks used (approx)

 

I know the data is still out there because a vxinstall will let me do a vxdisk -e list and it reveals the disk belong to rootdg.   Hopefully were are not toast.

 

devans3428
Level 4

Is there another form of communication you are willing to do(ie IM(googletalk, yahoo), phone, etc...)  Just thought i would ask.   If so i will put my outside email account on here and we can continue that way or we can continue like we are doing now.   Thanks for assistance...

Gaurav_S
Moderator
Moderator
   VIP    Certified

Data is still out there however its ASL which has caused issues to device names here.. & yes vxinstall will not touch out data region...

If you get vxconfigd to start , I would suggest to first exclude your ASL ...

# vxddladm excludearray libname=libvxDGCclariion.so

 

sorry m typing this thru mobile so bit slow..

 

G

devans3428
Level 4

I have tried to run vxinstall but didnt follow through with completion in fear the process would wipe out my public region where the data lives.   I really wasnt sure but i think vxinstall only touches the private region.  Your thought on whether i should try to run vxinstall.   My root drive is not part of vxvm.

I can run vxinstall and then in another window i am able to run vxdisk list and it displays this:

# vxdisk -e  list
DEVICE       TYPE      DISK         GROUP        STATUS       c#t#d#_NAME
Disk_0       sliced    disk01       rootdg       online       c10t500601613B241045d7s2
Disk_1       sliced    -            -            error        c10t500601613B241045d6s2
Disk_2       sliced    -            -            online       c10t500601613B241045d5s2
Disk_3       sliced    -            -            online       c10t500601613B241045d4s2
Disk_4       sliced    -            -            online       c10t500601613B241045d3s2
Disk_5       sliced    -            -            online       c10t500601613B241045d2s2
Disk_6       sliced    -            -            online       c10t500601613B241045d1s2
Disk_7       sliced    -            -            online       c10t500601613B241045d0s2
c1t0d0s2     sliced    -            -            error        c1t0d0s2
c1t1d0s2     sliced    -            -            error        c1t1d0s2

# vxdisk -o alldgs list
DEVICE       TYPE      DISK         GROUP        STATUS
Disk_0       sliced    disk01       rootdg       online
Disk_1       sliced    -            -            error
Disk_2       sliced    -           (rootdg)      online
Disk_3       sliced    -           (rootdg)      online
Disk_4       sliced    -           (rootdg)      online
Disk_5       sliced    -           (rootdg)      online
Disk_6       sliced    -           (rootdg)      online
Disk_7       sliced    -           (rootdg)      online
c1t0d0s2     sliced    -            -            error
c1t1d0s2     sliced    -            -            error
 

 

But HEY LOOK BELOW I WAS ABLE TO GET THE PATHS TO SHOW CORRECTLY!!!!!!.....So now the problem is getting vxconfigd to be enabled.
# vxdisk list Disk_0
Device:    Disk_0
devicetag: Disk_0
type:      sliced
hostid:    ncsun1
disk:      name=disk01 id=1302235001.1049.ncsun1
group:     name=rootdg id=1302234996.1025.ncsun1
flags:     online ready private autoconfig autoimport imported
pubpaths:  block=/dev/vx/dmp/Disk_0s4 char=/dev/vx/rdmp/Disk_0s4
privpaths: block=/dev/vx/dmp/Disk_0s3 char=/dev/vx/rdmp/Disk_0s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=629094400
private:   slice=3 offset=1 len=20479
update:    time=1302241535 seqno=0.17
headers:   0 248
configs:   count=1 len=15096
logs:      count=1 len=2287
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-015113[014865]: copy=01 offset=000231 enabled
 log      priv 015114-017400[002287]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   2
c10t500601613B241045d7s2        state=enabled
c16t500601603B241045d7s2        state=enabled
 

So the focus has turned from dmp to vxconfigd....

Gaurav_S
Moderator
Moderator
   VIP    Certified

so what does # vxdctl mode  shows now ?

 

devans3428
Level 4

WOW!!!!!!

# vxdctl mode
mode: enabled
 

So now the question is how do i get the disk back in rootdg.  The ones that show (rootdg) are the original disk that should be in rootdg.  Do i just perform vxdisksetup -i on all the disk that show (rootdg)?

# vxdisk -o alldgs list
DEVICE       TYPE      DISK         GROUP        STATUS
Disk_0       sliced    disk01       rootdg       online
Disk_1       sliced    -            -            error
Disk_2       sliced    -           (rootdg)      online
Disk_3       sliced    -           (rootdg)      online
Disk_4       sliced    -           (rootdg)      online
Disk_5       sliced    -           (rootdg)      online
Disk_6       sliced    -           (rootdg)      online
Disk_7       sliced    -           (rootdg)      online
c1t0d0s2     sliced    -            -            error
c1t1d0s2     sliced    -            -            error
# vxdisk -e list
DEVICE       TYPE      DISK         GROUP        STATUS       c#t#d#_NAME
Disk_0       sliced    disk01       rootdg       online       c10t500601613B241045d7s2
Disk_1       sliced    -            -            error        c10t500601613B241045d6s2
Disk_2       sliced    -            -            online       c10t500601613B241045d5s2
Disk_3       sliced    -            -            online       c10t500601613B241045d4s2
Disk_4       sliced    -            -            online       c10t500601613B241045d3s2
Disk_5       sliced    -            -            online       c10t500601613B241045d2s2
Disk_6       sliced    -            -            online       c10t500601613B241045d1s2
Disk_7       sliced    -            -            online       c10t500601613B241045d0s2
c1t0d0s2     sliced    -            -            error        c1t0d0s2
c1t1d0s2     sliced    -            -            error        c1t1d0s2
 

Gaurav_S
Moderator
Moderator
   VIP    Certified

NO, don't run vxdisksetup .. if offsets are given wrong you may landup in loosing all data..

Why we see less disks ? From the output of first post

 

fabric_0     sliced    -            -            error
fabric_1     sliced    -            -            error
fabric_2     sliced    disk05       rootdg       online
fabric_3     sliced    disk06       rootdg       online
fabric_4     sliced    disk07       rootdg       online
fabric_5     sliced    disk08       rootdg       online
fabric_6     sliced    disk09       rootdg       online
fabric_7     sliced    disk10       rootdg       online
fabric_8     sliced    -            -            error
fabric_9     sliced    -            -            error
fabric_10    sliced    -           (rootdg)      online
fabric_11    sliced    -           (rootdg)      online
fabric_12    sliced    -           (rootdg)      online
fabric_13    sliced    -           (rootdg)      online
fabric_14    sliced    -           (rootdg)      online
fabric_15    sliced    -           (rootdg)      online

 

now we see only

 

# vxdisk -e list
DEVICE       TYPE      DISK         GROUP        STATUS       c#t#d#_NAME
Disk_0       sliced    disk01       rootdg       online       c10t500601613B241045d7s2
Disk_1       sliced    -            -            error        c10t500601613B241045d6s2
Disk_2       sliced    -            -            online       c10t500601613B241045d5s2
Disk_3       sliced    -            -            online       c10t500601613B241045d4s2
Disk_4       sliced    -            -            online       c10t500601613B241045d3s2
Disk_5       sliced    -            -            online       c10t500601613B241045d2s2
Disk_6       sliced    -            -            online       c10t500601613B241045d1s2
Disk_7       sliced    -            -            online       c10t500601613B241045d0s2
c1t0d0s2     sliced    -            -            error        c1t0d0s2
c1t1d0s2     sliced    -            -            error        c1t1d0s2

 

why less disks ? moreover did u excluded the array library ?

we need to ensure that, the disks which were imported in rootdg before i.e

fabric_2     sliced    disk05       rootdg       online
fabric_3     sliced    disk06       rootdg       online
fabric_4     sliced    disk07       rootdg       online
fabric_5     sliced    disk08       rootdg       online
fabric_6     sliced    disk09       rootdg       online
fabric_7     sliced    disk10       rootdg       online

 

are the same ones now, which are shown as

 

Disk_0       sliced    disk01       rootdg       online
Disk_1       sliced    -            -            error
Disk_2       sliced    -           (rootdg)      online
Disk_3       sliced    -           (rootdg)      online
Disk_4       sliced    -           (rootdg)      online
Disk_5       sliced    -           (rootdg)      online
Disk_6       sliced    -           (rootdg)      online
Disk_7       sliced    -           (rootdg)      online

 

so problem is, rootdg is imported & deported too (yeah sounds crazy) ..but u can't deport rootdg & reimport it ... vxvm will expect the same disks to be present .

 

can u confirm if these are the same disks which were visible & imported before ?

 

G

devans3428
Level 4

What i did to get the paths to show correctly was vxddladm

# /etc/vx/diag.d/vxdmpinq /dev/rdsk/c10t500601613B241045d0s2

Inquiry for /dev/rdsk/c10t500601613B241045d0s2, evpd 0x0, page code 0x0
        Vendor id                        : DGC
        Product id                       : RAID 5
        Revision                         : 0428
        Serial Number                    : 0C0000D4FCCL
# /etc/vx/diag.d/vxdmpinq /dev/rdsk/c10t500601613B241045d1s2

Inquiry for /dev/rdsk/c10t500601613B241045d1s2, evpd 0x0, page code 0x0
        Vendor id                        : DGC
        Product id                       : RAID 5
        Revision                         : 0428
        Serial Number                    : 100000DC30CL
# vxddladm addjbod vid=DGC
# vxddladm listjbod
VID      PID              Opcode   Page Code   Page Offset      SNO length
==========================================================================
SEAGATE  ALL PIDs            18         -1         36           12
SUN      SESS01              18         -1         36           12
DGC      ALL PIDs            18         -1         36           12

 

# vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c1              OTHER_DISKS     ENABLED      OTHER_DISKS
c10             Disk            ENABLED      Disk
c16             Disk            ENABLED      Disk
 

devans3428
Level 4

Here is a copy of an old vxdisk list prior to the T3 SAN being removed.  I have output of various vxprint, vxdisk, etc... of previous setup.   You can see the fabric devices were part of rootdg

(root)> vxdisk -o alldgs list

DEVICE       TYPE      DISK         GROUP        STATUS

T30_0        sliced    disk01       rootdg       online

T30_1        sliced    disk02       rootdg       online

T31_0        sliced    disk03       rootdg       online

T31_1        sliced    disk04       rootdg       online

T32_0        sliced    rootdg00     rootdg       online

T32_1        sliced    rootdg01     rootdg       online

c1t0d0s2     sliced    -            -            error

c1t1d0s2     sliced    -            -            error

fabric_0     sliced    -            -            error

fabric_1     sliced    -            -            error

fabric_2     sliced    disk05       rootdg       online

fabric_3     sliced    disk06       rootdg       online

fabric_4     sliced    disk07       rootdg       online

fabric_5     sliced    disk08       rootdg       online

fabric_6     sliced    disk09       rootdg       online

fabric_7     sliced    disk10       rootdg       online

fabric_8     sliced    -            -            error

fabric_9     sliced    -            -            error

fabric_10    sliced    -           (rootdg)      online

fabric_11    sliced    -           (rootdg)      online

fabric_12    sliced    -           (rootdg)      online

fabric_13    sliced    -           (rootdg)      online

fabric_14    sliced    -           (rootdg)      online

fabric_15    sliced    -           (rootdg)      online

 Also prior vxdisk list -e

(root)> vxdisk -e list

DEVICE       TYPE      DISK         GROUP        STATUS       c#t#d#_NAME

T30_0        sliced    disk01       rootdg       online       c7t60020F20000104AE3EB2EF5A00058465d0s2

T30_1        sliced    disk02       rootdg       online       c7t60020F20000104AE3EB2EF4A000C3565d0s2

T31_0        sliced    disk03       rootdg       online       c7t60020F20000107DC3EB2F0310003CB97d0s2

T31_1        sliced    disk04       rootdg       online       c7t60020F20000107DC3EB2F021000BEB2Dd0s2

T32_0        sliced    rootdg00     rootdg       online       c7t60020F200000C2DC44EF5AF7000B560Dd0s2

T32_1        sliced    rootdg01     rootdg       online       c7t60020F200000C2DC44EF593900005A36d0s2

c1t0d0s2     sliced    -            -            error        c1t0d0s2

c1t1d0s2     sliced    -            -            error        c1t1d0s2

fabric_0     sliced    -            -            error        c10t500601613B241045d7s2

fabric_1     sliced    -            -            error        c10t500601613B241045d6s2

fabric_2     sliced    disk05       rootdg       online       c10t500601613B241045d5s2

fabric_3     sliced    disk06       rootdg       online       c10t500601613B241045d4s2

fabric_4     sliced    disk07       rootdg       online       c10t500601613B241045d3s2

fabric_5     sliced    disk08       rootdg       online       c10t500601613B241045d2s2

fabric_6     sliced    disk09       rootdg       online       c10t500601613B241045d1s2

fabric_7     sliced    disk10       rootdg       online       c10t500601613B241045d0s2

fabric_8     sliced    -            -            error        c16t500601603B241045d7s2

fabric_9     sliced    -            -            error        c16t500601603B241045d6s2

fabric_10    sliced    -            -            online       c16t500601603B241045d5s2

fabric_11    sliced    -            -            online       c16t500601603B241045d4s2

fabric_12    sliced    -            -            online       c16t500601603B241045d3s2

fabric_13    sliced    -            -            online       c16t500601603B241045d2s2

fabric_14    sliced    -            -            online       c16t500601603B241045d1s2

fabric_15    sliced    -            -            online       c16t500601603B241045d0s2

devans3428
Level 4

These disk were part of rootdg before as rootdg was the only diskgroup.  The disk that shows up correctly is a new disk that added with no worries about losing data.   The other disk that have (rootdg) have data on them.    So i am 100% sure they belong to rootdg.   We see less disk in output now because the extra disk we were seeing before came from the other path for disk 0-7.   Doing the vxddladm add DGC corrected my path issue which was the original problem.  So now vxdisk shows the correct number of disks.  So now when i do vxdisk list Disk_0 i see two paths instead of 1.  Next step?

g_lee
Level 6

devans3428,

In this comment ( https://www-secure.symantec.com/connect/forums/dynamic-multipath-using-emc-storage-veritas-35#comment-5410891 ), you mentioned:

"Well we decided since we mirrored the T3 plex with EMC plex under one volume and one disk group we would go ahead and remove the T3 plex and leave the EMC plex in place.  This would leave just the one EMC plex and no T3 volumes exist now."

Did you cleanly remove the T3 disks from the dg (vxdg -g <dg> rmdisk <disk>) before they were removed/unpresented from the OS?

Your original dg (on T3) appears to have had 6 disks. VxVM keeps 5 diskgroup configuration copies by default (may be 4 or 5 on 3.5, can't remember off the top of my head since it's a v old version!). Unless you've performed an explicit config change to place an active config copy every disk, this means only 5 disks will have an active dg config (or up to 5 if the dg had less than 5 disks). If you didn't remove the T3 disks cleanly (ie: at VxVM level) before removing physically and rebooting, it's possible these disks had the active copies, so this would be why there are no valid copies found on the EMC disks now.

If the disks weren't removed cleanly, then you may need to rebuild the config from previous output (if you'd saved it before), or worst case you may need to put the T3 disks back. Please confirm how the disks were removed from the OS so we can eliminate this as a possibility/determine how to proceed.

devans3428
Level 4

Yes i did vxdg -g rootdg rmdisk T3_device  and what was left in rootdg were the EMC luns.... 

Also i am past the valid copies now as vxdctl is in enable mode.   Now i just need to get the old disk back under rootdg. 

thanks