10-22-2020 03:45 AM
Dears,
I hope you're all well & safe.
We are using Veritas Infoscale 7.4 and we are having a 6 nodes cluster.
The output of the vxdisk list command on the first two nodes is showing the below:
DEVICE TYPE DISK GROUP STATUS
sba-emm-app-pro_vmdk0_0 auto:sliced sba-emm-app-pro_vmdk0_0 mmdbdg20 online shared
sba-emm-app-pro_vmdk0_1 auto:sliced sba-emm-app-pro_vmdk0_1 mmdata1dg20 online shared
sba-emm-app-pro_vmdk0_2 auto:sliced sba-emm-app-pro_vmdk0_2 mmdatadg20 online shared
sba-emm-app-pro_vmdk0_3 auto:LVM - - LVM
For the rest of the four remaining nodes, the output is the following:
DEVICE TYPE DISK GROUP STATUS
sba-emm-app-pro_vmdk0_0 auto:LVM - - LVM
sba-emm-app-pro_vmdk0_1 auto:sliced sba-emm-app-pro_vmdk0_0 mmdbdg20 online shared
sba-emm-app-pro_vmdk0_2 auto:sliced sba-emm-app-pro_vmdk0_1 mmdata1dg20 online shared
sba-emm-app-pro_vmdk0_3 auto:sliced sba-emm-app-pro_vmdk0_2 mmdatadg20 online shared
If you notice, the "DEVICE" output is inconsistent between the nodes.
Could you please help me understand why I can see this inconsistency? And is there a way to make them all consistent?
Thanks & Regards,
Karim Momtaz
10-23-2020 03:19 AM
read this tn https://www.veritas.com/support/en_US/article.100020472
and follow the instructions in the solution section.
although the tn was originally for an old version, the solution is still valid and easy to implement (harmless)
10-26-2020 06:32 AM
Hi Frank,
Thanks for your reply.
I tested the procedure mentioned in your reply on a 2 nodes cluster having the same mismatch:
SBA-EMM-APP-TEST1:/dev/vx/rdmp# vxdisk list
DEVICE TYPE DISK GROUP STATUS
sba-emm-app-tes_vmdk0_0 auto:LVM - - LVM
sba-emm-app-tes_vmdk0_1 auto:sliced sba-emm-app-tes_vmdk0_1 mmdatadg20 online shared
sba-emm-app-tes_vmdk0_2 auto:sliced sba-emm-app-tes_vmdk0_2 mmdbdg20 online shared
SBA-EMM-APP-TEST2:/etc/vx# vxdisk list
DEVICE TYPE DISK GROUP STATUS
sda auto:LVM - - LVM
sdb auto:sliced sba-emm-app-tes_vmdk0_1 mmdatadg20 online shared
sdc auto:sliced sba-emm-app-tes_vmdk0_2 mmdbdg20 online shared
I applied the procedure on the second node first then the first one but they are the same after doing it. I mean no changes were found. The exact commands I used:
1- hasys -freeze SBA-EMM-APP-TEST2
2- rm -f /dev/vx/dmp/*
3- rm -f /dev/vx/rdmp/*
4- mv /etc/vx/array.info /etc/vx/array.info.old
5- mv /etc/vx/disk.info /etc/vx/disk.info.old2
6- vxconfigd -k -x syslog
7- hasys -unfreeze SBA-EMM-APP-TEST2
Any idea why it didn't fix it as expected?
10-27-2020 03:15 AM
lets troubleshoot the issue further.
Please run the commans below on both nodes and post the output
1. vxdisk list sdb | grep -i disk | grep -i id
2. vxdisk list sdc | grep -i disk | grep -i id
3. vxdg list
where on the second node, you need to replace the disk names sdb and sdc which the actual disk names shown in vxdisk list output accordingly.
on node SBA-EMM-APP-TEST1, run the commands below:
a. vxddladm set namingscheme=osn
b. vxdisk -o alldgs list | grep -i shared
10-27-2020 04:19 AM
Hi again Frank,
Thanks for your reply.
One thing I tried before you reply, I tried to make the namingscheme as OSN on both nodes. Then I saw the output of the command vxdisk list. The two nodes are now consistently showing sda sdb & sdc. The question is: why the EBN is not showing the consistent device names? And is that normal? Is it preferrable to keep the naming scheme as OSN?
One more thing I noticed on another cluster I have, the device names are inconsistent however the Disks udid is the same:
SBA-OLM-APP-PROD1:/opt/OLM_Temp_Storage_Script# vxdisk list sba-olm-app-pro_vmdk0_0 | grep -i disk | grep -i id
disk: name=sba-olm-app-pro_vmdk0_0 id=1600956871.5.SBA-OLM-APP-PROD1
udid: VMware%5FVirtual%20disk%5Fvmdk%5F6000C291E64FE1586C96AFD3338D9E76
SBA-OLM-APP-PROD1:/opt/OLM_Temp_Storage_Script# vxdisk list sba-olm-app-pro_vmdk0_2 | grep -i disk | grep -i id
disk: name=sba-olm-app-pro_vmdk0_2 id=1600956917.8.SBA-OLM-APP-PROD1
udid: VMware%5FVirtual%20disk%5Fvmdk%5F6000C29CFF80C9A87F79A0962AF60D33
SBA-OLM-APP-PROD3:~# vxdisk list sba-olm-app-pro_vmdk0_1 | grep -i disk | grep -i id
disk: name=sba-olm-app-pro_vmdk0_0 id=1600956871.5.SBA-OLM-APP-PROD1
udid: VMware%5FVirtual%20disk%5Fvmdk%5F6000C291E64FE1586C96AFD3338D9E76
SBA-OLM-APP-PROD3:~# vxdisk list sba-olm-app-pro_vmdk0_2 | grep -i disk | grep -i id
disk: name=sba-olm-app-pro_vmdk0_2 id=1600956917.8.SBA-OLM-APP-PROD1
udid: VMware%5FVirtual%20disk%5Fvmdk%5F6000C29CFF80C9A87F79A0962AF60D33
Please find the outputs requested from the TEST1 & TEST2:
SBA-EMM-APP-TEST1:~# vxdisk list
DEVICE TYPE DISK GROUP STATUS
sba-emm-app-tes_vmdk0_0 auto:LVM - - LVM
sba-emm-app-tes_vmdk0_1 auto:sliced sba-emm-app-tes_vmdk0_1 mmdatadg20 online shared
sba-emm-app-tes_vmdk0_2 auto:sliced sba-emm-app-tes_vmdk0_2 mmdbdg20 online shared
SBA-EMM-APP-TEST1:~# vxdisk list sba-emm-app-tes_vmdk0_1 | grep -i disk | grep -i id
disk: name=sba-emm-app-tes_vmdk0_1 id=1594114694.8.SBA-EMM-APP-TEST1
udid: VMware%5FVirtual%20disk%5Fvmdk%5F6000C2961C52EA4CEE31062822D2DA25
SBA-EMM-APP-TEST1:~# vxdisk list sba-emm-app-tes_vmdk0_2 | grep -i disk | grep -i id
disk: name=sba-emm-app-tes_vmdk0_2 id=1594114629.5.SBA-EMM-APP-TEST1
udid: VMware%5FVirtual%20disk%5Fvmdk%5F6000C29BF1F6192A60A34641524FA1DE
SBA-EMM-APP-TEST1:~# vxdg list
NAME STATE ID
mmdbdg20 enabled,shared 1594114630.7.SBA-EMM-APP-TEST1
mmdatadg20 enabled,shared 1594114694.10.SBA-EMM-APP-TEST1
SBA-EMM-APP-TEST2:~# vxdisk list sdb | grep -i disk | grep -i id
disk: name=sba-emm-app-tes_vmdk0_1 id=1594114694.8.SBA-EMM-APP-TEST1
udid: VMware%5FVirtual%20disk%5FOTHER%5FDISKS%5FSBA-EMM-APP-TEST2%5F%2Fdev%2Fsdb
SBA-EMM-APP-TEST2:~# vxdisk list sdc | grep -i disk | grep -i id
disk: name=sba-emm-app-tes_vmdk0_2 id=1594114629.5.SBA-EMM-APP-TEST1
udid: VMware%5FVirtual%20disk%5FOTHER%5FDISKS%5FSBA-EMM-APP-TEST2%5F%2Fdev%2Fsdc
SBA-EMM-APP-TEST2:~# vxdg list
NAME STATE ID
mmdbdg20 enabled,shared 1594114630.7.SBA-EMM-APP-TEST1
mmdatadg20 enabled,shared 1594114694.10.SBA-EMM-APP-TEST1
SBA-EMM-APP-TEST1:~# vxddladm set namingscheme=osn
SBA-EMM-APP-TEST1:~# vxdisk -o alldgs list | grep -i shared
sdb auto:sliced sba-emm-app-tes_vmdk0_1 mmdatadg20 online shared
sdc auto:sliced sba-emm-app-tes_vmdk0_2 mmdbdg20 online shared
Thanks a lot for your help.
10-28-2020 04:18 AM
some of your questions are answered below:
Q1 why the EBN is not showing the consistent device names?
A1.very much dependent on the system device trees and the VxVM versions and ASL installed
Q2. is that normal?
A2. Yes
Q3. Is it preferrable to keep the naming scheme as OSN?
A3. It is recommended to use ebn. If you only look after a few systems and each system has less than 100 disks/luns, use obn is OK
To make the device names coonsisten, run vxddladm with attribute persistence (check the command man page for command syntax).
The most important thing is the device IDs, that is each node in the cluster sees the same shared devices.
10-29-2020 03:09 AM
forgot to mention one thing in the previous email - the two shared disks are TPD, right?
If the disks are TPD, you can run vxdmpadm command below
vxdmpadm setattr enclosure enclr_name tpdmode={native|pseudo}
to set the tpdmode per below:
tpdmode - Specifies the form of device names for devices that are controlled by third-party drivers (TPDs) whose coexistence is supported by an appropriate ASL. If set to pseudo (default), the TPD metadevice name is used. If set to native, the subpath with the smallest device number known to the operating system is used.