Multiple entries shown in vxdisk for the same EMC Clariion device

[Gao]

HW  :
Two Solaris 10 Update 9  cross connecting EMC Storage AX4-5

Setup:
Two nodes cluster running Veritas Storage Foundation Cluster File System/HA 5.1 SP1.
2 virtual disks are configured on EMC storage.

Problem:
1. vxdisk shows not only two EMC virtual disks (emcpower0 and emcpower1), but also the whole EMC storage device emc_clariion0_0.
2. Number of paths for emcpower0 or emcpower1 is 1 instead of 2.
Please note that the same configurations work OK on Solaris 9 + VCS 5.0 MP3.

bash-3.00# vxdisk -e list

DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR        

disk_0       auto:none      -            -           online invalid       c0t5000CCA00A767F58d0s2 -            

disk_1       auto:sliced    -            -           online               c0t5000CCA00AA97084d0s2 -            

emc_clariion0_0 auto:none      -            -           online invalid       c2t0d0           -            

emcpower0    auto:none      -            -           online invalid       emcpower0        -            

emcpower1    auto:none      -            -           online invalid       emcpower1        -            

bash-3.00# vxdisk path

SUBPATH                     DANAME               DMNAME       GROUP        STATE

c0t5000CCA00A767F58d0s2     disk_0               -            -            ENABLED

c0t5000CCA00AA97084d0s2     disk_1               -            -            ENABLED

c2t0d0                      emc_clariion0_0      -            -            ENABLED

c2t0d1                      emc_clariion0_0      -            -            DISABLED

emcpower0                   emcpower0            -            -            ENABLED

emcpower1                   emcpower1            -            -            ENABLED

bash-3.00# vxdisk list emc_clariion0_0

Device:    emc_clariion0_0

devicetag: emc_clariion0_0

type:      auto

info:      format=none

flags:     online ready private autoconfig invalid

pubpaths:  block=/dev/vx/dmp/emc_clariion0_0 char=/dev/vx/rdmp/emc_clariion0_0

guid:      -

udid:      INVALID

site:      -

Multipathing information:

numpaths:   2

c2t0d0          state=enabled

c2t0d1          state=disabled

bash-3.00# vxdisk list emcpower0

Device:    emcpower0

devicetag: emcpower0

type:      auto

info:      format=none

flags:     online ready private autoconfig invalid

pubpaths:  block=/dev/vx/dmp/emcpower0 char=/dev/vx/rdmp/emcpower0

guid:      -

udid:      DGC%5FRAID%205%5FSL7E4084300077%5F600601609AEB2500269D9F04F56BE011

site:      -

Multipathing information:

numpaths:   1

emcpower0       state=enabled

bash-3.00# format

Searching for disks...done

AVAILABLE DISK SELECTIONS:

       0. c0t5000CCA00A767F58d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>

          /scsi_vhci/disk@g5000cca00a767f58

       1. c0t5000CCA00AA97084d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>

          /scsi_vhci/disk@g5000cca00aa97084

       2. c1t0d0 <DGC-RAID 5-0223-1.13TB>

          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w500601684ba0449c,0

       3. c1t0d1 <DGC-RAID 5-0223-1.96TB>

          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w500601684ba0449c,1

       4. c2t0d0 <DGC-RAID 5-0223-1.13TB>

          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0/ssd@w500601604ba0449c,0

       5. c2t0d1 <DGC-RAID 5-0223-1.96TB>

          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0/ssd@w500601604ba0449c,1

       6. emcpower0a <DGC-RAID 5-0223-1.13TB>

          /pseudo/emcp@0

       7. emcpower1a <DGC-RAID 5-0223-1.96TB>

          /pseudo/emcp@1

Thanks in advance for help.

Best Regards,
Gao

[Gaurav_S]

Hello,

I am not really concerned about only path being shown for emcpower devices because they are managed by powerpath, so yes, you should only see one path beneath emcpower device. If you want to see the alternate paths for that device, you should use

# powermt display dev=all     (or dev=emcpower0a)

Now regarding other disks which are appearing as emc_clariion0_0,  was this disk added later ? anything specific was done for this device ? Also, in the format menu, the disks are being shown like internal disks with no FC name or wwn in it , do you know why it is like this ? Also, is this a new setup or was working before ?

Was powerdevices configured later to DMP ? can u paste below outputs:

# vxdmpadm listenclosure all

# vxdmpadm listctlr all

# vxddladm listexclude all

# vxdmpadm listexclude all

Gaurav

[Gao]

Thanks Gaurav. I was wrong on problem 1, and "# powermt display dev=all" shows that two paths are available for either emcpower0 or emcpower1.

The disk appearing as emc_clariion0_0 was not added later. Actually no specific was done for EMC storage. And you are right the disks are shown like internal disks yet I don't know the reasons.

This is not a new setup. Previously T5220 + Solaris 9 + VCS 5.0 MP3 works OK for years.
Now we've changed to T3 + Solaris 10 + VCS 5.1, and this extra disk emc_clariion0_0 appears in VXVM (causing disk group failure later on).

I'm OT at home, so I have only logs recorded earlier as below. I've also done Google searching for 3 days with no luck. Hope that you can see what's wrong from below info. Many thanks!

bash-3.00# vxdmpadm listenclosure all

ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT 

=======================================================================================

pp_emc_clariion0  PP_EMC_CLARiiON  SL7E4084300077       CONNECTED    A/A         2

disk              Disk           DISKS                CONNECTED    Disk        2

emc_clariion0_0    EMC_CLARiiON     ALUA    .......       <= This line is written from memory since I lost the log

bash-3.00# vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME

=====================================================

c0              Disk            ENABLED      disk

c2              EMC_CLARiiON    ENABLED      emc_clariion1

emcp            PP_EMC_CLARiiON  ENABLED      pp_emc_clariion0

bash-3.00# vxddladm listsupport all  | grep -i emc

libvxemc.so         EMC                 SYMMETRIX

libvxpp.so          EMC, DGC            All

bash-3.00# vxdmpadm listexclude all

Devices excluded from VxVM:

--------------------------

Paths : None

Controllers : None

VID:PID : None

Devices excluded from multipathing by vxdmp:

-------------------------------------------

Paths : None

Controllers : None

VID:PID : None

Pathgroups : None

----------

BRs,
Gao

[Gaurav_S · Accepted solution]

Ok, well looking at above outputs, I can not clearly say that emc_clariion0_0 is an invalid/stale entry or no .. reason being:

in the vxdmpadm listctlr all output above I see that c2 is connected to emc_clariion1 .. point to note is, status is shown as "connected" ...  now, if we see the devices in vxdisk list output, they also appearing from "c2" ..

Also, if we carefully observe the path for c2 devices, they are coming from "qlc@0,1/fp"  however all c1 disks are at "qlc@0/fp"  (notice the bold characters)

so here is what I would suggest:

-- Get the storage checked , Lun masking/settings done correctly, make sure zoning is done correctly at SAN switches & that too connections reported to c2 controller are fine from storage end. Also storage settings are as per symantec recommended (see HCL for 5.1)

http://www.symantec.com/docs/TECH74012

-- Make sure OS part is correct, like luxadm -e port is reporting FC connected, cfgadm reports the controller configured, if not try configuring/reconfiguring

-- Lastly, if all above appears in order, try following below technote to cleanup the device tree, there are two choices in technote, one is reconfigure reboot while another is online method, give a go on online method first...

http://www.symantec.com/docs/TECH78463

Hope this helps

Gaurav

[Gao]

I will go to my employer's site and try tomorrow. Thanks anyway.

BRs,
Gao