Forum Discussion

Gao's avatar
Gao
Level 2
14 years ago

Multiple entries shown in vxdisk for the same EMC Clariion device

HW  :
Two Solaris 10 Update 9  cross connecting EMC Storage AX4-5

Setup:
Two nodes cluster running Veritas Storage Foundation Cluster File System/HA 5.1 SP1.
2 virtual disks are configured on EMC storage.

Problem:
1. vxdisk shows not only two EMC virtual disks (emcpower0 and emcpower1), but also the whole EMC storage device emc_clariion0_0.
2. Number of paths for emcpower0 or emcpower1 is 1 instead of 2.
Please note that the same configurations work OK on Solaris 9 + VCS 5.0 MP3.

 

bash-3.00# vxdisk -e list
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR        
disk_0       auto:none      -            -           online invalid       c0t5000CCA00A767F58d0s2 -            
disk_1       auto:sliced    -            -           online               c0t5000CCA00AA97084d0s2 -            
emc_clariion0_0 auto:none      -            -           online invalid       c2t0d0           -            
emcpower0    auto:none      -            -           online invalid       emcpower0        -            
emcpower1    auto:none      -            -           online invalid       emcpower1        -            
 
bash-3.00# vxdisk path
SUBPATH                     DANAME               DMNAME       GROUP        STATE
c0t5000CCA00A767F58d0s2     disk_0               -            -            ENABLED
c0t5000CCA00AA97084d0s2     disk_1               -            -            ENABLED
c2t0d0                      emc_clariion0_0      -            -            ENABLED
c2t0d1                      emc_clariion0_0      -            -            DISABLED
emcpower0                   emcpower0            -            -            ENABLED
emcpower1                   emcpower1            -            -            ENABLED
 
bash-3.00# vxdisk list emc_clariion0_0
Device:    emc_clariion0_0
devicetag: emc_clariion0_0
type:      auto
info:      format=none
flags:     online ready private autoconfig invalid
pubpaths:  block=/dev/vx/dmp/emc_clariion0_0 char=/dev/vx/rdmp/emc_clariion0_0
guid:      -
udid:      INVALID
site:      -
Multipathing information:
numpaths:   2
c2t0d0          state=enabled
c2t0d1          state=disabled
 
bash-3.00# vxdisk list emcpower0
Device:    emcpower0
devicetag: emcpower0
type:      auto
info:      format=none
flags:     online ready private autoconfig invalid
pubpaths:  block=/dev/vx/dmp/emcpower0 char=/dev/vx/rdmp/emcpower0
guid:      -
udid:      DGC%5FRAID%205%5FSL7E4084300077%5F600601609AEB2500269D9F04F56BE011
site:      -
Multipathing information:
numpaths:   1
emcpower0       state=enabled
 
 
bash-3.00# format
Searching for disks...done
 
 
AVAILABLE DISK SELECTIONS:
       0. c0t5000CCA00A767F58d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>
          /scsi_vhci/disk@g5000cca00a767f58
       1. c0t5000CCA00AA97084d0 <SUN300G cyl 46873 alt 2 hd 20 sec 625>
          /scsi_vhci/disk@g5000cca00aa97084
       2. c1t0d0 <DGC-RAID 5-0223-1.13TB>
          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w500601684ba0449c,0
       3. c1t0d1 <DGC-RAID 5-0223-1.96TB>
          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0/fp@0,0/ssd@w500601684ba0449c,1
       4. c2t0d0 <DGC-RAID 5-0223-1.13TB>
          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0/ssd@w500601604ba0449c,0
       5. c2t0d1 <DGC-RAID 5-0223-1.96TB>
          /pci@400/pci@1/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0/ssd@w500601604ba0449c,1
       6. emcpower0a <DGC-RAID 5-0223-1.13TB>
          /pseudo/emcp@0
       7. emcpower1a <DGC-RAID 5-0223-1.96TB>
          /pseudo/emcp@1
 
 
Thanks in advance for help.

Best Regards,
Gao

business continuity
Integration
Storage Foundation
Troubleshooting
  • Gaurav_S's avatar
    Gaurav_S
    14 years ago

    Ok, well looking at above outputs, I can not clearly say that emc_clariion0_0 is an invalid/stale entry or no .. reason being:

    in the vxdmpadm listctlr all output above I see that c2 is connected to emc_clariion1 .. point to note is, status is shown as "connected" ...  now, if we see the devices in vxdisk list output, they also appearing from "c2" ..

    Also, if we carefully observe the path for c2 devices, they are coming from "qlc@0,1/fp"  however all c1 disks are at "qlc@0/fp"  (notice the bold characters)

    so here is what I would suggest:

    -- Get the storage checked , Lun masking/settings done correctly, make sure zoning is done correctly at SAN switches & that too connections reported to c2 controller are fine from storage end. Also storage settings are as per symantec recommended (see HCL for 5.1)

    http://www.symantec.com/docs/TECH74012

    -- Make sure OS part is correct, like luxadm -e port is reporting FC connected, cfgadm reports the controller configured, if not try configuring/reconfiguring

    -- Lastly, if all above appears in order, try following below technote to cleanup the device tree, there are two choices in technote, one is reconfigure reboot while another is online method, give a go on online method first...

    http://www.symantec.com/docs/TECH78463

     

     

    Hope this helps

     

    Gaurav

  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    14 years ago

    Hello,

    I am not really concerned about only path being shown for emcpower devices because they are managed by powerpath, so yes, you should only see one path beneath emcpower device. If you want to see the alternate paths for that device, you should use

    # powermt display dev=all     (or dev=emcpower0a)

     

    Now regarding other disks which are appearing as emc_clariion0_0,  was this disk added later ? anything specific was done for this device ? Also, in the format menu, the disks are being shown like internal disks with no FC name or wwn in it , do you know why it is like this ? Also, is this a new setup or was working before ?

    Was powerdevices configured later to DMP ? can u paste below outputs:

    # vxdmpadm listenclosure all

    # vxdmpadm listctlr all

    # vxddladm listexclude all

    # vxdmpadm listexclude all

     

    Gaurav

  • Gao's avatar
    Gao
    Level 2
    14 years ago

    Thanks Gaurav. I was wrong on problem 1, and "# powermt display dev=all" shows that two paths are available for either emcpower0 or emcpower1.

    The disk appearing as emc_clariion0_0 was not added later. Actually no specific was done for EMC storage. And you are right the disks are shown like internal disks yet I don't know the reasons.

    This is not a new setup. Previously T5220 + Solaris 9 + VCS 5.0 MP3 works OK for years.
    Now we've changed to T3 + Solaris 10 + VCS 5.1, and this extra disk emc_clariion0_0 appears in VXVM (causing disk group failure later on).

    I'm OT at home, so I have only logs recorded earlier as below. I've also done Google searching for 3 days with no luck. Hope that you can see what's wrong from below info. Many thanks!

    bash-3.00# vxdmpadm listenclosure all
    ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT 
    =======================================================================================
    pp_emc_clariion0  PP_EMC_CLARiiON  SL7E4084300077       CONNECTED    A/A         2
    disk              Disk           DISKS                CONNECTED    Disk        2
    emc_clariion0_0    EMC_CLARiiON     ALUA    .......       <= This line is written from memory since I lost the log
     

    bash-3.00# vxdmpadm listctlr all
    CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME

     

    =====================================================
    c0              Disk            ENABLED      disk
    c2              EMC_CLARiiON    ENABLED      emc_clariion1
    emcp            PP_EMC_CLARiiON  ENABLED      pp_emc_clariion0
     
    bash-3.00# vxddladm listsupport all  | grep -i emc
    libvxemc.so         EMC                 SYMMETRIX
    libvxpp.so          EMC, DGC            All
     
    bash-3.00# vxdmpadm listexclude all
    Devices excluded from VxVM:
    --------------------------
    Paths : None
     
    Controllers : None
     
    VID:PID : None
     
    Devices excluded from multipathing by vxdmp:
    -------------------------------------------
    Paths : None
     
    Controllers : None
     
    VID:PID : None
     
    Pathgroups : None
    ----------
     
     

    BRs,
    Gao

  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    14 years ago

    Ok, well looking at above outputs, I can not clearly say that emc_clariion0_0 is an invalid/stale entry or no .. reason being:

    in the vxdmpadm listctlr all output above I see that c2 is connected to emc_clariion1 .. point to note is, status is shown as "connected" ...  now, if we see the devices in vxdisk list output, they also appearing from "c2" ..

    Also, if we carefully observe the path for c2 devices, they are coming from "qlc@0,1/fp"  however all c1 disks are at "qlc@0/fp"  (notice the bold characters)

    so here is what I would suggest:

    -- Get the storage checked , Lun masking/settings done correctly, make sure zoning is done correctly at SAN switches & that too connections reported to c2 controller are fine from storage end. Also storage settings are as per symantec recommended (see HCL for 5.1)

    http://www.symantec.com/docs/TECH74012

    -- Make sure OS part is correct, like luxadm -e port is reporting FC connected, cfgadm reports the controller configured, if not try configuring/reconfiguring

    -- Lastly, if all above appears in order, try following below technote to cleanup the device tree, there are two choices in technote, one is reconfigure reboot while another is online method, give a go on online method first...

    http://www.symantec.com/docs/TECH78463

     

     

    Hope this helps

     

    Gaurav

  • Gao's avatar
    Gao
    Level 2
    14 years ago

    I will go to my employer's site and try tomorrow. Thanks anyway.

     

    BRs,
    Gao