05-18-2016 11:26 PM
Hi,
I'm trying to install Veritas Storage Foundation Cluster File System HA 6.0.3 to 6 KVM RHEL guests.
I installed RHEL6.4 and created 3 virtual fencing disks using qemu-img. I added them as SCSI disks to the node and did the vxdisksetup for them. Now when I tried to configure fencing, I get the error below.
I'm wondering if anyone has installed cluster file system into KVM guest environment and could tell me how to add virtual fencing disks so that fencing configuration would be successful.
This is what I tried on the KVM:
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/home/VM/VMImages/IOFencing1.img'/>
<target dev='sda' bus='scsi'/>
<shareable/>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/home/VM/VMImages/IOFencing2.img'/>
<target dev='sdb' bus='scsi'/>
<shareable/>
<alias name='scsi0-0-0-1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/home/VM/VMImages/IOFencing3.img'/>
<target dev='sdc' bus='scsi'/>
<shareable/>
<alias name='scsi0-0-0-2'/>
<address type='drive' controller='0' bus='0' target='0' unit='2'/>
</disk>
Here is how the disks look like:
[root@node1 installsfcfsha601-201605190528YQa]# vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
disk_0 auto:cdsdisk - - online
disk_1 auto:cdsdisk - - online
disk_2 auto:cdsdisk - - online
disk_3 auto:none - - online invalid
disk_4 auto:none - - online invalid
vda auto:none - - online invalid
[root@node1 installsfcfsha601-201605190528YQa]#
[root@node1 installsfcfsha601-201605190528YQa]# cat /etc/vxfentab
#
# /etc/vxfentab:
# DO NOT MODIFY this file as it is generated by the
# VXFEN rc script from the file /etc/vxfendg.
#
/dev/vx/rdmp/disk_0 QEMU%5FQEMU%20HARDDISK%5FDISKS%5Fdrive-scsi0-0-0-0
/dev/vx/rdmp/disk_1 QEMU%5FQEMU%20HARDDISK%5FDISKS%5Fdrive-scsi0-0-0-1
/dev/vx/rdmp/disk_2 QEMU%5FQEMU%20HARDDISK%5FDISKS%5Fdrive-scsi0-0-0-2
[root@node1 installsfcfsha601-201605190528YQa]#
The error I get:
kernel: I/O Fencing DISABLED!VXFEN INFO V-11-1-35 Fencing driver going into RUNNING state
kernel: GAB INFO V-15-1-20032 Port b closed
kernel: GAB INFO V-15-1-20229 Client VxFen deiniting GAB API
kernel: VXFEN INFO V-11-1-36 VxFEN configured at protocol version 30
kernel: GAB INFO V-15-1-20230 Client VxFen inited GAB API with handle ffff8803b8288ac0
kernel: GAB INFO V-15-1-20036 Port b[VxFen (refcount 2)] gen 93aa25 membership 0
kernel: GAB INFO V-15-1-20038 Port b[VxFen (refcount 2)] gen 93aa25 k_jeopardy ;12345
kernel: GAB INFO V-15-1-20040 Port b[VxFen (refcount 2)] gen 93aa25 visible ;12345
kernel: VXFEN WARNING V-11-1-12 Potentially a
kernel: preexisting split-brain.
kernel: Dropping out of cluster.
kernel: Refer to user documentation for
kernel: steps required to clear preexisting
kernel: split-brain.
05-19-2016 12:19 AM
So I guess it seems that the virtual SCSI disks are not SCSI-3 compliant. How do I create SCSI-3 compliant virtual scsi disks?
[root@node1 ~]# vxfentsthdw
Veritas vxfentsthdw version 6.0.300.000-GA Linux
The utility vxfentsthdw works on the two nodes of the cluster.
The utility verifies that the shared storage one intends to use is
configured to support I/O fencing. It issues a series of vxfenadm
commands to setup SCSI-3 registrations on the disk, verifies the
registrations on the disk, and removes the registrations from the disk.
******** WARNING!!!!!!!! ********
THIS UTILITY WILL DESTROY THE DATA ON THE DISK!!
Do you still want to continue : [y/n] (default: n) y
The logfile generated for vxfentsthdw is /var/VRTSvcs/log/vxfen/vxfentsthdw.log.16323
Enter the first node of the cluster:
node1
Enter the second node of the cluster:
node2
Enter the disk name to be checked for SCSI-3 PGR on node node1 in the format:
for dmp: /dev/vx/rdmp/sdx
for raw: /dev/sdx
Make sure it is the same disk as seen by nodes node1 and node2
/dev/vx/rdmp/disk_0
Enter the disk name to be checked for SCSI-3 PGR on node node2 in the format:
for dmp: /dev/vx/rdmp/sdx
for raw: /dev/sdx
Make sure it is the same disk as seen by nodes node1 and node2
/dev/vx/rdmp/disk_0
***************************************************************************
Testing node1 /dev/vx/rdmp/disk_0 node2 /dev/vx/rdmp/disk_0
Evaluate the disk before testing ........................... Pre-existing keys
RegisterIgnoreKeys on disk /dev/vx/rdmp/disk_0 from node node1 ....... Failed
RegisterIgnoreKeys to disk failed on node node1.
Disk is not SCSI-3 compliant on node node1.
Execute the utility vxfentsthdw again and if failure persists contact
the vendor for support in enabling SCSI-3 persistent reservations
Removing test keys and temporary files, if any...
[root@node1 ~]#