Forum Discussion

Juha_2's avatar
Juha_2
Level 2
9 years ago

How to add virtual fencing disk to KVM guest

Hi,

I'm trying to install Veritas Storage Foundation Cluster File System HA 6.0.3 to 6 KVM RHEL guests.

I installed RHEL6.4 and created 3 virtual fencing disks using qemu-img. I added them as SCSI disks to the node and did the vxdisksetup for them. Now when I tried to configure fencing, I get the error below.

I'm wondering if anyone has installed cluster file system into KVM guest environment and could tell me how to add virtual fencing disks so that fencing configuration would be successful.

This is what I tried on the KVM:

    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/home/VM/VMImages/IOFencing1.img'/>
      <target dev='sda' bus='scsi'/>
      <shareable/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/home/VM/VMImages/IOFencing2.img'/>
      <target dev='sdb' bus='scsi'/>
      <shareable/>
      <alias name='scsi0-0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/home/VM/VMImages/IOFencing3.img'/>
      <target dev='sdc' bus='scsi'/>
      <shareable/>
      <alias name='scsi0-0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>

Here is how the disks look like:

[root@node1 installsfcfsha601-201605190528YQa]# vxdisk -o alldgs list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:cdsdisk    -            -            online
disk_1       auto:cdsdisk    -            -            online
disk_2       auto:cdsdisk    -            -            online
disk_3       auto:none       -            -            online invalid
disk_4       auto:none       -            -            online invalid
vda          auto:none       -            -            online invalid
[root@node1 installsfcfsha601-201605190528YQa]# 

[root@node1 installsfcfsha601-201605190528YQa]# cat /etc/vxfentab 
#
# /etc/vxfentab: 
# DO NOT MODIFY this file as it is generated by the
# VXFEN rc script from the file /etc/vxfendg.
#
/dev/vx/rdmp/disk_0 QEMU%5FQEMU%20HARDDISK%5FDISKS%5Fdrive-scsi0-0-0-0
/dev/vx/rdmp/disk_1 QEMU%5FQEMU%20HARDDISK%5FDISKS%5Fdrive-scsi0-0-0-1
/dev/vx/rdmp/disk_2 QEMU%5FQEMU%20HARDDISK%5FDISKS%5Fdrive-scsi0-0-0-2
[root@node1 installsfcfsha601-201605190528YQa]# 

The error I get:

kernel:      I/O Fencing DISABLED!VXFEN INFO V-11-1-35 Fencing driver going into RUNNING state
kernel: GAB INFO V-15-1-20032 Port b closed
kernel: GAB INFO V-15-1-20229 Client VxFen deiniting GAB API
kernel: VXFEN INFO V-11-1-36 VxFEN configured at protocol version 30
kernel: GAB INFO V-15-1-20230 Client VxFen inited GAB API with handle ffff8803b8288ac0
kernel: GAB INFO V-15-1-20036 Port b[VxFen (refcount 2)] gen   93aa25 membership 0
kernel: GAB INFO V-15-1-20038 Port b[VxFen (refcount 2)] gen   93aa25 k_jeopardy ;12345
kernel: GAB INFO V-15-1-20040 Port b[VxFen (refcount 2)] gen   93aa25    visible ;12345
kernel: VXFEN WARNING V-11-1-12 Potentially a
kernel:        preexisting split-brain.
kernel:        Dropping out of cluster.
kernel:        Refer to user documentation for
kernel:        steps required to clear preexisting
kernel:        split-brain.

 

business continuity
Linux
Storage Foundation for Windows
  • Juha_2's avatar
    Juha_2
    Level 2
    9 years ago

    So I guess it seems that the virtual SCSI disks are not SCSI-3 compliant. How do I create SCSI-3 compliant virtual scsi disks?

     

    [root@node1 ~]# vxfentsthdw

    Veritas vxfentsthdw version 6.0.300.000-GA Linux


    The utility vxfentsthdw works on the two nodes of the cluster.
    The utility verifies that the shared storage one intends to use is
    configured to support I/O fencing.  It issues a series of vxfenadm
    commands to setup SCSI-3 registrations on the disk, verifies the
    registrations on the disk, and removes the registrations from the disk.


    ******** WARNING!!!!!!!! ********

    THIS UTILITY WILL DESTROY THE DATA ON THE DISK!! 

    Do you still want to continue : [y/n] (default: n) y
    The logfile generated for vxfentsthdw is /var/VRTSvcs/log/vxfen/vxfentsthdw.log.16323

    Enter the first node of the cluster:
    node1
    Enter the second node of the cluster:
    node2

    Enter the disk name to be checked for SCSI-3 PGR on node node1 in the format:
            for dmp: /dev/vx/rdmp/sdx
            for raw: /dev/sdx
    Make sure it is the same disk as seen by nodes node1 and node2
    /dev/vx/rdmp/disk_0

    Enter the disk name to be checked for SCSI-3 PGR on node node2 in the format:
            for dmp: /dev/vx/rdmp/sdx
            for raw: /dev/sdx
    Make sure it is the same disk as seen by nodes node1 and node2
    /dev/vx/rdmp/disk_0

    ***************************************************************************

    Testing node1 /dev/vx/rdmp/disk_0 node2 /dev/vx/rdmp/disk_0

    Evaluate the disk before testing  ........................... Pre-existing keys
    RegisterIgnoreKeys on disk /dev/vx/rdmp/disk_0 from node node1 ....... Failed

    RegisterIgnoreKeys to disk  failed on node node1.
            Disk  is not SCSI-3 compliant on node node1.
            Execute the utility vxfentsthdw again and if failure persists contact
            the vendor for support in enabling SCSI-3 persistent reservations


    Removing test keys and temporary files, if any...
    [root@node1 ~]#