I have installed VCS HA clusterware on solaris 10 sparc arch. I have configured vxfen in SCSI3 mode.
What I observe is that all the disks which are part of co-ordinated DG have SCSI3 reservation but my data
disk doesn't have any reservation. My apprehension is that if VCS nodes doesnt do reservation how VCS is
going to protect the disk from outside world, in fact I tried to write on my data disk from a linux box which is not
the part of the cluster?
I am wondering if I have done the configuration correct?
Please find some of command output below which might help to analyze if there is some
configuration mistake:
===============
bash-3.00# vxfenadm -d
Fencing Protocol Version: 201
Fencing Mode: SCSI3
Fencing SCSI3 Disk Policy: raw
Cluster Members:
0 (Solaris146)
* 1 (Solaris148)
RFSM State Information:
node 0 in state 8 (running)
node 1 in state 8 (running)
bash-3.00# hagrp -state
#Group Attribute System Value
groupw State Solaris146 |OFFLINE|
groupw State Solaris148 |ONLINE|
bash-3.00# hares -state
#Resource Attribute System Value
fodg State Solaris146 OFFLINE
fodg State Solaris148 ONLINE
fofs State Solaris146 OFFLINE
fofs State Solaris148 ONLINE
bash-3.00#
Once the DG is imported by VCS, you will see the registraiton & reservation of only 1 node (in case of a failover diskgroup) ... However if its a Shared diskgroup (CVM diskgroup), then you would see registration & reservation of both nodes....
Yes you are right, VCS diskgroup agent imports the diskgroup with specific options ( -o groupreserve) then only reservations are placed.... If you have a normal filesystem where diskgroup was not imported with above options, then it is NOT protected by VCS IOFencing..... however you can configure that FS in VCS ... so basically you just loose the split brain protection (which is real purpose of fencing)....
In Veritas IOFencing, coordinator disks get only registration keys however data disks will have both registration keys & reservation keys...
To check Registration keys..
# /sbin/vxfenadm -g /dev/rdsk/<cxtxdx>
To check reservation keys..
# /sbin/vxfenadm -r /dev/rdsk/<cxtxdx>
Since you have configured VCS as well, did the diskgroups were imported by VCS ? because VCS will import the diskgroups with reservations... then only you will see reservation keys...
Also, just running of vxfen module doesn't indicate that fencing is active....
Does your main.cf had an entry "UseFence = SCSI3" ? If yes, then only cluster would be using IOfencing...
As you mentioned you tried writing via a linux box .... how did it go ? did it succeed ? Only that node can write to the disk who has registration keys & wrtie-exclusive reservation key ... your linux box should get an error while attempting write...
As you said I could see the registration as well as reservation once VCS imported the disk group.
But at a time I only see one registration (from the node where DG is currently online) on the disk while I was expecting two registration keys as it is two node cluster.
One more question as we started seeing SCSI keys after VCS imported VxVM diskgroup does that means if I were to create a VCS HA file-system on a raw disk then the disk would not be protected by VCS?
Yes, in main.cf I have added entry "UseFence = SCSI3", I believe VCS will not start if there is discrepency in main.cf and vxfen.
From linux box it succeeded at the first time because VCS has not imported the diskgroup but later it failed.
But I dont see any reservation or registration for the raw disk (without VXVM diskgroup) which are part of a hares of Mount type.
The essential components of VCS I/O fencing are as follows:
1.> LLT, GAB, and VXFEN drivers
2.> Coordinator disks with SCSI-3 Persistent Reservations (PR)
3.> Veritas Volume Manager (VxVM) and Dynamic Multipathing (DMP)
4.> VCS I/O fencing data disks using CVM/DMP and VCS Disk Group agent
As you could see in point number 3 that volume manager is required, that means that it assumes that disks / luns / arrays are being managed by volume manager. For any other disk (without VXVM diskgroup), you would not see fencing keys in it.
Once the DG is imported by VCS, you will see the registraiton & reservation of only 1 node (in case of a failover diskgroup) ... However if its a Shared diskgroup (CVM diskgroup), then you would see registration & reservation of both nodes....
Yes you are right, VCS diskgroup agent imports the diskgroup with specific options ( -o groupreserve) then only reservations are placed.... If you have a normal filesystem where diskgroup was not imported with above options, then it is NOT protected by VCS IOFencing..... however you can configure that FS in VCS ... so basically you just loose the split brain protection (which is real purpose of fencing)....
