Problem using DMP for ASM

[Alistair_Veit1]

Hi,

Platform is RHEL6.4 on x64, Storage Foundation 6.0.1 with 6.0.3 patches applied.

[root@rhel64_ora1 ~]# lsb_release -i -r
Distributor ID: RedHatEnterpriseServer
Release:        6.4

[root@rhel64_ora1 ~]# vxlicrep -s | tail -3
   Product Name                        = VERITAS Volume Manager
   License Type                        = PERMANENT

[root@rhel64_ora1 ~]# rpm -qa | grep vxvm
VRTSvxvm-6.0.300.000-RHEL6.x86_64

[root@rhel64_ora1 ~]# vxdmpadm gettune dmp_native_support
            Tunable               Current Value  Default Value
------------------------------    -------------  -------------
dmp_native_support                       on              off
 

I'm trying to configure DMP's raw devices for use by ASM. Symantec's documentation states that the command "vxdmpraw" should create devices under /dev/raw, but on my system this doesn't work. Using disk /dev/sdd with a single partition as recommended by Oracle for ASM as a test:

[root@rhel64_ora1 ~]# fdisk -l /dev/sdd

Disk /dev/sdd: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xf2ea9f0a

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1         652     5237158+  83  Linux
 

[root@rhel64_ora1 ~]# ls -l /dev/vx/rdmp/sdd*
brw-------. 1 root root 201, 16 Jul  2 10:19 /dev/vx/rdmp/sdd
brw-------. 1 root root 201, 17 Jul  2 10:19 /dev/vx/rdmp/sdd1
brw-------. 1 root root 201, 26 Jul  2 10:19 /dev/vx/rdmp/sdd10
brw-------. 1 root root 201, 27 Jul  2 10:19 /dev/vx/rdmp/sdd11
brw-------. 1 root root 201, 28 Jul  2 10:19 /dev/vx/rdmp/sdd12
brw-------. 1 root root 201, 29 Jul  2 10:19 /dev/vx/rdmp/sdd13
brw-------. 1 root root 201, 30 Jul  2 10:19 /dev/vx/rdmp/sdd14
brw-------. 1 root root 201, 31 Jul  2 10:19 /dev/vx/rdmp/sdd15
brw-------. 1 root root 201, 18 Jul  2 10:19 /dev/vx/rdmp/sdd2
brw-------. 1 root root 201, 19 Jul  2 10:19 /dev/vx/rdmp/sdd3
brw-------. 1 root root 201, 20 Jul  2 10:19 /dev/vx/rdmp/sdd4
brw-------. 1 root root 201, 21 Jul  2 10:19 /dev/vx/rdmp/sdd5
brw-------. 1 root root 201, 22 Jul  2 10:19 /dev/vx/rdmp/sdd6
brw-------. 1 root root 201, 23 Jul  2 10:19 /dev/vx/rdmp/sdd7
brw-------. 1 root root 201, 24 Jul  2 10:19 /dev/vx/rdmp/sdd8
brw-------. 1 root root 201, 25 Jul  2 10:19 /dev/vx/rdmp/sdd9
 

[root@rhel64_ora1 ~]# vxdmpraw enable grid oinstall 660 sdd

  VxVM vxdmpraw INFO V-5-2-6157
                Device enabled : sdd
Cannot locate block device '/dev/vx/rdmp/sdds1' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds2' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds3' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds4' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds5' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds6' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds7' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds8' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds9' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds10' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds11' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds12' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds13' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds14' (No such file or directory)
Cannot locate block device '/dev/vx/rdmp/sdds15' (No such file or directory)

  VxVM vxdmpraw INFO V-5-2-6155
                Please set ASM_DISKSTRING to '/dev/raw/*' or '/dev/vx/rdmp/*' or
                '/dev/vx/dmp/*' for Oracle ASM to work on DMP devices.

[root@rhel64_ora1 ~]# ls -l /dev/vx/rdmp/sdd*
brw-rw----. 1 grid oinstall 201, 16 Jul  2 10:19 /dev/vx/rdmp/sdd
brw-------. 1 root root     201, 17 Jul  2 10:19 /dev/vx/rdmp/sdd1
brw-------. 1 root root     201, 26 Jul  2 10:19 /dev/vx/rdmp/sdd10
brw-------. 1 root root     201, 27 Jul  2 10:19 /dev/vx/rdmp/sdd11
brw-------. 1 root root     201, 28 Jul  2 10:19 /dev/vx/rdmp/sdd12
brw-------. 1 root root     201, 29 Jul  2 10:19 /dev/vx/rdmp/sdd13
brw-------. 1 root root     201, 30 Jul  2 10:19 /dev/vx/rdmp/sdd14
brw-------. 1 root root     201, 31 Jul  2 10:19 /dev/vx/rdmp/sdd15
brw-------. 1 root root     201, 18 Jul  2 10:19 /dev/vx/rdmp/sdd2
brw-------. 1 root root     201, 19 Jul  2 10:19 /dev/vx/rdmp/sdd3
brw-------. 1 root root     201, 20 Jul  2 10:19 /dev/vx/rdmp/sdd4
brw-------. 1 root root     201, 21 Jul  2 10:19 /dev/vx/rdmp/sdd5
brw-------. 1 root root     201, 22 Jul  2 10:19 /dev/vx/rdmp/sdd6
brw-------. 1 root root     201, 23 Jul  2 10:19 /dev/vx/rdmp/sdd7
brw-------. 1 root root     201, 24 Jul  2 10:19 /dev/vx/rdmp/sdd8
brw-------. 1 root root     201, 25 Jul  2 10:19 /dev/vx/rdmp/sdd9

[root@rhel64_ora1 ~]# more /etc/vx/.vxdmprawdev
sdd 500 500 0660
sdds10 500 500 0660 /dev/raw/raw10
sdds11 500 500 0660 /dev/raw/raw11
sdds12 500 500 0660 /dev/raw/raw12
sdds13 500 500 0660 /dev/raw/raw13
sdds14 500 500 0660 /dev/raw/raw14
sdds1 500 500 0660 /dev/raw/raw1
sdds15 500 500 0660 /dev/raw/raw15
sdds2 500 500 0660 /dev/raw/raw2
sdds3 500 500 0660 /dev/raw/raw3
sdds4 500 500 0660 /dev/raw/raw4
sdds5 500 500 0660 /dev/raw/raw5
sdds6 500 500 0660 /dev/raw/raw6
sdds7 500 500 0660 /dev/raw/raw7
sdds8 500 500 0660 /dev/raw/raw8
sdds9 500 500 0660 /dev/raw/raw9


[root@rhel64_ora1 ~]# ls /dev/raw
rawctl
 

So, the command "vxdmpraw enable grid oinstall 660 sdd" above doesn't put anything into /dev/raw as recommended for ASM's use.

The code in the vxdmpraw script is written to expect all partitions in /dev/vx/rdmp to have the name "$devname"s"${part}". Note the hard-coded "s" which doesn't match the actual partition names previously created by VxVM/DMP.

Admittedly, this test is inside an VirtualBox VM with virtual disks (I've tried virtualised SATA, SCSI and SAS disks with the same result) - do we need physical disk presentation for this to work properly, or is this a bug, or am I missing something?

Any views or experience on this?

Best Regards, Alistair.

[Douglas_Snyder · Accepted solution]

Hello Alistair,

Please check page 33 of the DMP Administrator’s guide.  There are a few pre-requisites and steps that need to be run in order to ensure ASM can see the disks.  This may help.

To make DMP devices visible to ASM

1 From ASM, make sure ASM_DISKSTRING is set to the value /dev/raw/*.

The value for ASM_DISKSTRING is /dev/vx/rdmp/*.

SQL> show parameter ASM_DISKSTRING;

NAME TYPE VALUE

-------------------- ----------- ---------------

asm_diskstring string /dev/vx/rdmp/*

2 As root user, enable DMP devices for use with ASM.

# vxdmpraw enable usernamegroupname [devicename ...]

For example:

# vxdmpraw enable oracle dba eva4k6k0_1

3 From ASM, confirm that ASM can see these new devices.

SQL> select name,path,header_status from v$asm_disk;

NAME PATH HEADER_STATUS

---------------------------------------------

... ....... ....

/dev/vx/rdmp/eva4k6k0_1 CANDIDATE

... .......

Hello Alistar