Dynamic multipath using EMC storage Veritas 3.5

OS can see both paths seems something is with the ASL....Oh and let me add I am not using navisphere or powerpath....

# luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
  Node WWN:50020f200000bd4f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F200000C2DC44EF5AF7000B560Dd0s2
  Node WWN:50020f200000bd4f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F200000C2DC44EF593900005A36d0s2
  Node WWN:50020f200001062f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000104AE3EB2EF4A000C3565d0s2
  Node WWN:50020f200001062f  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000104AE3EB2EF5A00058465d0s2
  Node WWN:50020f2000010c33  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000107DC3EB2F021000BEB2Dd0s2
  Node WWN:50020f2000010c33  Device Type:Disk device
    Logical Path:/dev/rdsk/c7t60020F20000107DC3EB2F0310003CB97d0s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d0s2
    Logical Path:/dev/rdsk/c16t500601603B241045d0s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d1s2
    Logical Path:/dev/rdsk/c16t500601603B241045d1s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d2s2
    Logical Path:/dev/rdsk/c16t500601603B241045d2s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d3s2
    Logical Path:/dev/rdsk/c16t500601603B241045d3s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d4s2
    Logical Path:/dev/rdsk/c16t500601603B241045d4s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d5s2
    Logical Path:/dev/rdsk/c16t500601603B241045d5s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d6s2
    Logical Path:/dev/rdsk/c16t500601603B241045d6s2
  Node WWN:50060160bb201045  Device Type:Disk device
    Logical Path:/dev/rdsk/c10t500601613B241045d7s2
    Logical Path:/dev/rdsk/c16t500601603B241045d7s2
#

Hello,

So its version 4.0 for vxvm, can you confirm the exact version ?

# pkginfo -l VRTSvxvm

Also, I see that you have ASL however somehow I believe either ASL is in excluded state or else its not claiming the devices...

Can you  paste some outputs:

# vxdmpadm listexclude all

# ls -l /etc/vx/*exclude*

if you find any exclude file, paste the contents of them...

Also, can you attach output of:

# /etc/vx/diag.d/vxasldebug    (if path is wrong, check in other directories for same command, not sure of exact path in 4.0)

Paths not being shown & vxdisk list showing as "fabric" both are relating to ASL issue only .. Tell me this also:

1. is this a new installation ? was it working before ?

2. whether ASL was installed manually or it came pre-installed ?

Gaurav

Hi Gaurav,

Sorry i thought i mention the version in my previous but i see i didnt.   Its 3.5 see below:

# pkginfo -l VRTSvxvm
   PKGINST:  VRTSvxvm
      NAME:  VERITAS Volume Manager, Binaries
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  3.5,REV=06.21.2002.23.14
   BASEDIR:  /
    VENDOR:  VERITAS Software
      DESC:  Virtual Disk Subsystem
    PSTAMP:  VERITAS-3.5s_PointPatch1.3:26-July-2002
  INSTDATE:  Dec 30 2003 20:16
   HOTLINE:  800-342-0652
     EMAIL:  support@veritas.com
    STATUS:  completely installed
     FILES:      600 installed pathnames
                  21 shared pathnames
                   9 linked files
                  76 directories
                 337 executables
              159005 blocks used (approx)

# vxdmpadm listexclude all
Devices excluded from VxVM:
--------------------------
Paths : None

Controllers : None

VID:PID : None

Devices excluded from multipathing by vxdmp:
-------------------------------------------
Paths : None

VID:PID : None

Pathgroups : None

# ls -l /etc/vx/*exclude*
-rw-r--r--   1 root     other         59 Dec 31  2003 /etc/vx/vxdmp.exclude
-rw-r--r--   1 root     other         59 Dec 31  2003 /etc/vx/vxvm.exclude
# cat /etc/vx/vxdmp.exclude
exclude_all 0
paths
#
controllers
#
product
#
pathgroups
#
# cat /etc/vx/vxvm.exclude
exclude_all 0
paths
#
controllers
#
product
#
pathgroups
 

Output for asldebug is attached in gzip format

This is a new installation of using EMC storage.  Before the storage was using T3 SAN.   The ASL library was installed via pkgadd just two days ago.  The T3 are using mpxio for multipathing.  

Here is a T3 disk its using mpxio as you can see by the long naming scheme...not concerned with the T3:

# vxdisk list T30_0
Device:    T30_0
devicetag: T30_0
type:      sliced
hostid:    ncsun1
disk:      name=disk01 id=1072877353.1041.nc1
group:     name=rootdg id=1072877341.1025.nc1
flags:     online ready private autoconfig autoimport imported
pubpaths:  block=/dev/vx/dmp/T30_0s4 char=/dev/vx/rdmp/T30_0s4
privpaths: block=/dev/vx/dmp/T30_0s3 char=/dev/vx/rdmp/T30_0s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1000595456
private:   slice=3 offset=1 len=57343
update:    time=1302107429 seqno=0.343
headers:   0 248
configs:   count=1 len=42303
logs:      count=1 len=6409
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-042320[042072]: copy=01 offset=000231 enabled
 log      priv 042321-048729[006409]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c7t60020F20000104AE3EB2EF5A00058465d0s2 state=enabled   type=primary
 

Here is EMC:

# vxdisk list fabric_2
Device:    fabric_2
devicetag: fabric_2
type:      sliced
hostid:    ncsun1
disk:      name=disk05 id=1302111549.6037.ncsun1
group:     name=rootdg id=1072877341.1025.nc1
info:      privoffset=1
flags:     online ready private autoconfig autoimport imported
pubpaths:  block=/dev/vx/dmp/fabric_2s4 char=/dev/vx/rdmp/fabric_2s4
privpaths: block=/dev/vx/dmp/fabric_2s3 char=/dev/vx/rdmp/fabric_2s3
version:   2.2
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=4 offset=0 len=1048494080
private:   slice=3 offset=1 len=32511
update:    time=1302111558 seqno=0.5
headers:   0 248
configs:   count=1 len=23969
logs:      count=1 len=3631
Defined regions:
 config   priv 000017-000247[000231]: copy=01 offset=000000 enabled
 config   priv 000249-023986[023738]: copy=01 offset=000231 enabled
 log      priv 023987-027617[003631]: copy=01 offset=000000 enabled
Multipathing information:
numpaths:   1
c10t500601613B241045d5s2        state=enabled
 

Ok, first surprise to me that its a new installation & you have installed 3.5 which is End of Life Ages ago .. Even 4.0  & in last OCt even 4.1 was end of life .... so any particular reason you are using such an old version ? 

I would very strongly recommend you to install latest version (5.1SP1 preferrable) .. I am saying this because even if your 3.5 setup gets to work, Symantec won't support it on any case unless your organization have support exception with them...

Ok coming to original problem .. my suspection was right ... you have ASL installed however it doesn't claim the Luns..  from the attached output:

checking asl library outputs
----------------------------------------------------------

**********************************************************
                   libvxDGCclariion.so                                     
**********************************************************

    libvxDGCclariion.so:vendor_info()
    VID                                    : DGC
    PID                                    : CLARiiON
    ATYPE                                  : A/P
    ANAME                                  : EMC_CLARiiON

    libvxDGCclariion.key()
    ASL Name                               : libvxDGCclariion.so
    Feature Needed                         : 95
    VxVM Version Needed                    : 32

    Device                                 : /dev/rdsk/c10t500601613B241045d0s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0C0000D4FCCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d1s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 100000DC30CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d2s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 140000E59FCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d3s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 160000E5A4CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d4s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0E0000D843CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d5s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 120000E16ECL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d6s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 5A00004614CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c10t500601613B241045d7s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 6000004CC9CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d0s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0C0000D4FCCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d1s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 100000DC30CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d2s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 140000E59FCL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d3s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 160000E5A4CL
    libvxDGCclariion.so:claim_device()     : ERROR

    Device                                 : /dev/rdsk/c16t500601603B241045d4s2
    Vendor Identification                  : DGC    
    Product Identification                 : RAID 5         
    Revision                               : 0428
    Serial Number                          : 0E0000D843CL
    libvxDGCclariion.so:claim_device()     : ERROR

you can see all disks in error state ... so now questions are:

-- are you sure you installed correct version of ASL which is supported by 3.5 ?

-- did you reboot the server after ASL installation ? I doubt if 3.5 will accept ASL changes online ?

--again, if you have any latest version of vxvm, it will have clariion ASL inbuilt & will pickup your devices automatically. ...

Think over it ..

G

I may have mislead you...this is not a new install ...the only new thing is adding the EMC clariion tray to the server.   We are not able to upgrade veritas at this time so trying to make this work as is with the new addition of the EMC tray.

I installed from here:

-- are you sure you installed correct version of ASL which is supported by 3.5 ?

http://www.symantec.com/business/support/index?page=content&id=TECH22609

Seems to be the correct one.... 3.5 solaris

-- did you reboot the server after ASL installation ? I doubt if 3.5 will accept ASL changes online ?

I have rebooted a couple of times along with doing reboot reconfigure

--again, if you have any latest version of vxvm, it will have clariion ASL inbuilt & will pickup your devices automatically. ...

Not able to upgrade because not worth the cost as customer is going away in a year so really trying to get this to work on 3.5

Thanks for the quick response....

Well we decided since we mirrored the T3 plex with EMC plex under one volume and one disk group we would go ahead and remove the T3 plex and leave the EMC plex in place.  This would leave just the one EMC plex and no T3 volumes exist now.  Upon reboot vxconfigd does not start because of message below

# vxconfigd
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_2, fabric_10
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_3, fabric_11
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_4, fabric_12
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_5, fabric_13
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_6, fabric_14
vxvm:vxconfigd: NOTICE: Unable to resolve duplicate diskid.
        Please refer to release notes and admin guide for possible action/solution.
        Following are the disks with duplicate diskid:
        Vendor: DGC Product: RAID - fabric_7, fabric_15
vxvm:vxconfigd: ERROR: enable failed: Error in disk group configuration copies
        Unexpected kernel error in configuration update; transactions are disabled.

Any idea how to resolve this one?   I think it all is an issue with the DGC library as you have stated.

Thanks
 

well true ..

try out these steps:

# rm /dev/vx/dmp/*

# rm /dev/vx/rdmp/*

# rm /dev/rdsk/*

# rm /dev/rdsk/*

# devfsadm -Cv

# /sbin/vxconfigd -k -m disable

check if vxconfigd started

# vxdctl mode   (see if it returns disabled)

Try to enable it..

# vxdctl enable

Also, can u paste output of:

pkginfo -l CLR-ASL

Gaurav

Did all the steps above and the results are below....

# vxdctl enable
vxvm:vxconfigd: ERROR: enable failed: Error in disk group configuration copies
        No valid disk found containing disk group; transactions are disabled.
vxvm:vxdctl: ERROR: enable failed: Error in disk group configuration copies
# pkginfo -l CLR-ASL
   PKGINST:  CLR-ASL
      NAME:  EMC CLARIION Array Support Library
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  1.04,REV=07.25.2002.17.36
   BASEDIR:  /etc/vx
    VENDOR:  EMC Corp
      DESC:  EMC CLARIION Array Support Library
    PSTAMP:  VERSION-1.04:14-February-2002
  INSTDATE:  Apr 07 2011 18:00
   HOTLINE:  800-782-4362
    STATUS:  completely installed
     FILES:        4 installed pathnames
                   2 shared pathnames
                   2 directories
                   2 executables
                  68 blocks used (approx)

I know the data is still out there because a vxinstall will let me do a vxdisk -e list and it reveals the disk belong to rootdg.   Hopefully were are not toast.

Is there another form of communication you are willing to do(ie IM(googletalk, yahoo), phone, etc...)  Just thought i would ask.   If so i will put my outside email account on here and we can continue that way or we can continue like we are doing now.   Thanks for assistance...

Data is still out there however its ASL which has caused issues to device names here.. & yes vxinstall will not touch out data region...

If you get vxconfigd to start , I would suggest to first exclude your ASL ...

# vxddladm excludearray libname=libvxDGCclariion.so

sorry m typing this thru mobile so bit slow..

G