cancel
Showing results for 
Search instead for 
Did you mean: 

"vxfenadm -i" reports different serial no for same disk on different server

khemmerl
Level 5

Hello everyone,

I'm following the steps in the Veritas Cluster Server Installation Guide Solaris 5.1, Chapter 7 Configuring VCS Clusters for Data Integrity the section Verifying that the nodes have access to the same disk.  The instructions are to run the "vxfenadm -i diskpath" command and confirm that the serial number is equivalent between the two nodes.  All disks are actually LUNs from my SAN and all report the same Vendor ID, Product ID and Revision but not all my disks are reporting the same serial number.  Below is a table of my disks and the serial number that get reported.  The first 26 characters of each serial number are the same so I have removed them to make the table a bit clearer.

Disk

Node 1 Serial No

Node 2 Serial No

/dev/rdsk/c3t50001FE150109D38d1s0

160000

160000

/dev/rdsk/c3t50001FE150109D38d2s0

600000

190000

/dev/rdsk/c3t50001FE150109D38d3s0

190000

310000

/dev/rdsk/c3t50001FE150109D38d4s0

310000

360000

/dev/rdsk/c3t50001FE150109D38d5s0

360000

3B0000

/dev/rdsk/c3t50001FE150109D38d6s0

3B0000

600000

As you can see, disk D38d2 from node 1 shows the serial number that matches disk D38d6 from node 2. My question is: How to I correct the configuration so that the same disk names on each node actually point to the same LUNs and return the same serial numbers?

Thanks in advance for any help.

Ken

1 ACCEPTED SOLUTION

Accepted Solutions

khemmerl
Level 5

It turns out that it's OK if the same disk name on on different servers accesses different LUNs on the SAN so long as the Veritas device names on the different servers access the same LUNs.  The path is:

Device > Disk > LUN

So long as the same device accesses the same LUN between nodes, it doesn't really matter what the name of the disk is.  Below is the information from my system.  Notice that the serial numbers match between nodes which means the same device name is using the same LUN on each node even though the disk name is different.

Node 1   Node 2
Serial # Disk Device Disk Serial #
160000 c3t50001FE150109D38d1s2 eva80000_0 c3t50001FE150109D38d1s2 160000
190000 c3t50001FE150109D38d3s2 eva80000_1 c3t50001FE150109D38d2s2 190000
310000 c3t50001FE150109D38d4s2 eva80000_2 c3t50001FE150109D38d3s2 310000
360000 c3t50001FE150109D38d5s2 eva80000_3 c3t50001FE150109D38d4s2 360000
3B0000 c3t50001FE150109D38d6s2 eva80000_4 c3t50001FE150109D38d5s2 3B0000
600000 c3t50001FE150109D38d2s2 eva80000_5 c3t50001FE150109D38d6s2 600000

Ken

View solution in original post

11 REPLIES 11

khemmerl
Level 5

The order of the third column within /etc/vx/disk.info seems to be messed up.  The 3rd, 4th and 5th columns of the disk.info files from Node 1 are:

0x4d80030    0x2    eva80000_0
0x4d80048    0x2    eva80000_1
0x4d80040    0x2    eva80000_2
0x4d80020    0x2    eva80000_3
0x4d80028    0x2    eva80000_4
0x4d80038    0x2    eva80000_5

The same columns from disk.info on node 2 are:
       
0x4d80040    0x2    eva80000_0
0x4d80048    0x2    eva80000_1
0x4d80028    0x2    eva80000_2
0x4d80038    0x2    eva80000_3
0x4d80030    0x2    eva80000_4
0x4d80020    0x2    eva80000_5

Can anyone tell me the best way to correct this info?

Ken

khemmerl
Level 5

More information:

Node 1: vxdisk -e list
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR
eva80000_0   auto:cdsdisk   -            -           online               c4t50001FE150109D39d1s2 -
eva80000_1   auto:cdsdisk   -            -           online               c4t50001FE150109D3Dd3s2 -
eva80000_2   auto:cdsdisk   -            -           online               c4t50001FE150109D3Dd4s2 -
eva80000_3   auto:cdsdisk   -            -           online               c4t50001FE150109D3Bd5s2 -
eva80000_4   auto:cdsdisk   -            -           online               c4t50001FE150109D39d6s2 -
eva80000_5   auto:cdsdisk   -            -           online               c4t50001FE150109D39d2s2 -

Node 2: vxdisk -e list
DEVICE       TYPE           DISK        GROUP        STATUS               OS_NATIVE_NAME   ATTR
eva80000_0   auto:cdsdisk   -            -           online               c4t50001FE150109D39d1s2 -
eva80000_1   auto:cdsdisk   -            -           online               c4t50001FE150109D39d2s2 -
eva80000_2   auto:cdsdisk   -            -           online               c4t50001FE150109D3Fd3s2 -
eva80000_3   auto:cdsdisk   -            -           online               c4t50001FE150109D3Fd4s2 -
eva80000_4   auto:cdsdisk   -            -           online               c4t50001FE150109D3Dd5s2 -
eva80000_5   auto:cdsdisk   -            -           online               c4t50001FE150109D39d6s2 -

Notice on Node 2 how the disks are listed in order (1 to 6) but on Node 1 the disks are listed in a messed-up order (1, 3-6, 2).  Again, any idea how to fixe this?  I'm assuming that if I can fix this order, the mis-matched serial number issue will be resolved.

Ken

Gaurav_S
Moderator
Moderator
   VIP    Certified

Hi Ken,

this happens depending on how OS device tree was built up, the order in which OS tree scans the disk, same is developed for veritas device tree as well..

To fix this, you will need to use parameter from vxddladm,

vxddladm set namingscheme=ebn persistence=yes use_avid=yes

in above, ebn is enclosure based naming, avid is array volume ID, so your Luns will be scanned & assigned the names depending on array volume ID (which is same for all nodes) .. so device names should be consistent ..

 

hope this helps

 

G

khemmerl
Level 5

I ran the command "vxddladm set namingscheme=ebn persistence=yes use_avid=yes" on both servers - it did not give any output.  I then rebooted both servers and checked the serial numbers returned by "vxfenadm -i /dev/rdsk/<disk_name>". 

Nothing has changed.  The one disk that reported the same serial number on both nodes is still correct and all other disks are mis-matched.

Do you have any other suggestions?

Ken

TonyGriffiths
Level 6
Employee Accredited Certified

Regarding the avid command, one possibility is that AVID (Array Volume ID) is not available with that array/ASL as it requires vendor specfic knowledge/interaction coded into the library

 

If not available, VxVM will have to auto generate the disk ids _1, _2 etc.

 

khemmerl
Level 5

Our SAN is an HP EVA 8000.

Is there a way to override the assignment of disk IDs?

khemmerl
Level 5

FYI: I've opened a case with Symantec support and have generated and uploaded vxexplorer files from each of my servers for their review.  I'll post updates here as they occur.

Ken

khemmerl
Level 5

It turns out that it's OK if the same disk name on on different servers accesses different LUNs on the SAN so long as the Veritas device names on the different servers access the same LUNs.  The path is:

Device > Disk > LUN

So long as the same device accesses the same LUN between nodes, it doesn't really matter what the name of the disk is.  Below is the information from my system.  Notice that the serial numbers match between nodes which means the same device name is using the same LUN on each node even though the disk name is different.

Node 1   Node 2
Serial # Disk Device Disk Serial #
160000 c3t50001FE150109D38d1s2 eva80000_0 c3t50001FE150109D38d1s2 160000
190000 c3t50001FE150109D38d3s2 eva80000_1 c3t50001FE150109D38d2s2 190000
310000 c3t50001FE150109D38d4s2 eva80000_2 c3t50001FE150109D38d3s2 310000
360000 c3t50001FE150109D38d5s2 eva80000_3 c3t50001FE150109D38d4s2 360000
3B0000 c3t50001FE150109D38d6s2 eva80000_4 c3t50001FE150109D38d5s2 3B0000
600000 c3t50001FE150109D38d2s2 eva80000_5 c3t50001FE150109D38d6s2 600000

Ken

TonyGriffiths
Level 6
Employee Accredited Certified

Hi

Look up the "assign names" option in the man page for vxddladm.

You should be able to use vxgetdmpnames to produce an output of your current mappings, edit it then use it in conjunction with the assign names option.

There should be a KM article on this, will see if I can find it and post it here

cheers

khemmerl
Level 5

I'd be interested in seeing the KM on this however I'm comfortable enough that I'll continue on with my installation.  FYI:  I don't see "vxgetdmpnames" on my installation so there may be an additional install required to get this to work.

 > vxgetdmpnames
bash: vxgetdmpnames: command not found 

Ken

TonyGriffiths
Level 6
Employee Accredited Certified

Hi,

Below is a SORT link to the 5.1SP1/Solaris vxvm admin guide (html) showing the topic of interest

https://sort.symantec.com/public/documents/sfha/5.1sp1/solaris/productguides/html/vxvm_admin/ch04s07.htm

The vxgetdmpnames utility shoul be in /etc/vx/bin, it is only in the recent releases.

Also found this KM article

http://www.symantec.com/docs/TECH69648

 

cheers