Not possible to create disk group in Cluster Storage Foundation HA whith EMC CX4-480

[Fabien_Monext]

Hello everybody,

I have the following configurations in order to test VCS in a ESX environment: 

Symantec software installer :Storage Foundation and High Availability Solutions 5.1 SP1

3 Nodes (Linux RHEL 5 64 bits ) in a VMware ESX environement 4.1 . I want to create a CP server  in this server (VMware didn't support scsi3 reservation)  , so i need to create a disk group in this cluster before.

For this disk group I present 3 disks in raw direct mapping of the EMC CX4-480 (Flare 30 ) san storage. The luns are correctly detected by the os , but not correctly detected by veritas volume manager.  For the ESX I present the luns with this settings: CLARION OPEN , ALU Active/Active Fail over mode 4.

I read : Veritas Storage Foundation and High Availability Solutions 5.1, 5.1 PR1, 5.1 SP1, and VirtualStore 5.1 SP1 PR3 Hardware Compatibility List

Device/Family Mode SF SF HA SFCFS/SFCFS Oracle RAC/SF Oracle RAC Interface Advanced Features

CLARiiON CX4 series [1] [4] [5] [6] A/A-A Yes Yes, Fencing Yes, Fencing Fibre Channel, iSCSI Advanced Reporting, SAN Boot, Thin Reclamation
CLARiiON CX4 series [1] [3] [5] [6] A/P-F Yes Yes, Fencing Yes, Fencing Fibre Channel, iSCSI Advanced Reporting, Thin Reclamation

1. NDU operations are supported, please consult with EMC support for exact procedures of performing NDU on the array.

4. A minimum flare code version 26 or above is required to support this mode.

5. The management tools must be used to report physically allocated space.
6. Thin Reclamation is supported with this array except on HP-UX platform; the minimum array firmware FLARE 29 is required, but to obtain the maximum reclamation
benefits, FLARE 30 is highly recommended; for more detail, please refer to EMC Powerlink Primus Doc ID# emc233231: CLARiiON Thin LUN Space Reclamation. Thin
Reclamation is not supported with PowerPath.

Ithink I respect all the prerequisite for my configuration. Could you please help me to solve this problem.

Thanks in adavance,

Fabien.

[root@vcsym3 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          16      128488+  83  Linux
/dev/sda2              17         538     4192965   83  Linux
/dev/sda3             539        1060     4192965   83  Linux
/dev/sda4            1061        2610    12450375    5  Extended
/dev/sda5            1061        1321     2096451   82  Linux swap / Solaris
/dev/sda6            1322        1582     2096451   83  Linux
/dev/sda7            1583        2610     8257378+  8e  Linux LVM

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdd doesn't contain a valid partition table
[root@vcsym3 ~]#
[root@vcsym3 ~]#
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxdisk list -e
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR  
emc_clariion0_120 auto           -            -           error                sdb              lun RAID_5
emc_clariion0_121 auto           -            -           error                sdc              lun RAID_5
emc_clariion0_122 auto           -            -           error                sdd              lun RAID_5
sda          auto:none      -            -           online invalid       sda              -     
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c1              EMC_CLARiiON    ENABLED      emc_clariion0
c0              OTHER_DISKS     ENABLED      other_disks
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxdmpadm getsubpaths ctlr=c1
NAME         STATE[A]   PATH-TYPE[M] DMPNODENAME  ENCLR-TYPE   ENCLR-NAME   ATTRS
================================================================================
sdb          ENABLED(A)  SECONDARY    emc_clariion0_120  EMC_CLARiiON  emc_clariion0    -
sdc          ENABLED(A)  SECONDARY    emc_clariion0_121  EMC_CLARiiON  emc_clariion0    -
sdd          ENABLED(A)  SECONDARY    emc_clariion0_122  EMC_CLARiiON  emc_clariion0    -
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxdmpadm listenclosure all
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO      STATUS       ARRAY_TYPE     LUN_COUNT
=======================================================================================
other_disks       OTHER_DISKS    OTHER_DISKS          CONNECTED    OTHER_DISKS   1
emc_clariion0     EMC_CLARiiON   CKM00105100057       CONNECTED    ALUA        3
[root@vcsym3 ~]#
[root@vcsym3 ~]#  vxddladm listexclude all

The Diskarrays excluded
-----------------------

Based on Library names:
-----------------------

Based on VID, PID Combination:
------------------------------
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxddladm listsupport all|grep -i emc
libvxpp.so          EMC, DGC            All
libvxemc.so         EMC                 SYMMETRIX
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxddladm listsupport libname=libvxpp.so
ATTR_NAME           ATTR_VALUE
=================================================================================================
LIBNAME             libvxpp.so
VID                 EMC, DGC
PID                 All
ARRAY_TYPE          A/A
ARRAY_NAME          PP_EMC, PP_EMC_CLARiiON
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxddladm listsupport libname=libvxemc.so
ATTR_NAME           ATTR_VALUE
=================================================================================================
LIBNAME             libvxemc.so
VID                 EMC
PID                 SYMMETRIX
ARRAY_TYPE          A/A
ARRAY_NAME          EMC
[root@vcsym3 ~]#
[root@vcsym3 ~]# vxdisk -o alldgs list
DEVICE       TYPE            DISK         GROUP        STATUS
emc_clariion0_120 auto            -            -            error
emc_clariion0_121 auto            -            -            error
emc_clariion0_122 auto            -            -            error
sda          auto:none       -            -            online invalid
[root@vcsym3 ~]# vxdisk list emc_clariion0_120
Device:    emc_clariion0_120
devicetag: emc_clariion0_120
type:      auto
flags:     error private autoconfig
pubpaths:  block=/dev/vx/dmp/emc_clariion0_120 char=/dev/vx/rdmp/emc_clariion0_120
guid:      -
udid:      DGC%5FRAID%205%5FCKM00105100057%5F6006016002302B009C0D360953B4E011
site:      -
Multipathing information:
numpaths:   1
sdb             state=enabled   type=secondary
[root@vcsym3 ~]#

[joseph_dangelo]

Fabien,

You may want to take a look at the following whitepaper.  It gives a pretty good overview of the support features with SF when installed in guest with VMware ESX.

https://www-secure.symantec.com/connect/articles/veritas-storage-foundation-vmware-esx-environment-l...

When you say VxVM is not properly detecting the luns, I am not quite following you. It would appear that the disks are visible and simply need to have a partition table added using fdisk and then they can be initialized. 

fdisk /dev//sdb

Write n for create partition

Write w for write partition table

Click p to check the created partition

From this point you can create your diskgroup and initialize your devices.

Hope this helps,

Joe D

[Marianne · Accepted solution]

Initialize disks, then add to diskgroup:

vxdisksetup -i emc_clariion0_120

vxdg init <dgname> vxdg init dg1 <diskname>=emc_clariion0_120

[Fabien_Monext]

Thanks for your respones.

Yesterday i read docs , which indicate to initialise disk before. But i did nt have the command vxdisksetup in my path.

So I search in my system , and in found this binaries in etc ...

I initialise the disk with this command and the disk are now online:

/etc/vx/bin/vxdisksetup -i emc_clariion0_120
/etc/vx/bin/vxdisksetup -i emc_clariion0_121
/etc/vx/bin/vxdisksetup -i emc_clariion0_122

[root@vcsym3 ~]# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
emc_clariion0_120 auto:cdsdisk    -            -            online
emc_clariion0_121 auto:cdsdisk    -            -            online
emc_clariion0_122 auto:cdsdisk    -            -            online
sda          auto:none       -            -            online invalid
[root@vcsym3 ~]#

Now I can create the disk group:

vxdg init cps_dg emc_clariion0_120 emc_clariion0_121

[root@vcsym3 ~]# vxdg -q list cps_dg
Group:     cps_dg
dgid:      1311668446.29.vcsym3
import-id: 1024.28
flags:     cds
version:   160
alignment: 8192 (bytes)
ssb:            on
autotagging:    on
detach-policy: global
dg-fail-policy: dgdisable
copies:    nconfig=default nlog=default
config:    seqno=0.1028 permlen=51360 free=51356 templen=2 loglen=4096
config disk emc_clariion0_120 copy 1 len=51360 state=clean online
config disk emc_clariion0_121 copy 1 len=51360 state=clean online
log disk emc_clariion0_120 copy 1 len=4096
log disk emc_clariion0_121 copy 1 len=4096
[root@vcsym3 ~]#

Thanks for you quick responses and the resolution of this case.

Fabien.