Forum Discussion

ajit_jain's avatar
ajit_jain
Level 2
15 years ago
Solved

SCSI reservation on data disk

Hello All,

I have installed VCS HA clusterware on solaris 10 sparc arch. I have configured vxfen in SCSI3 mode.
What I observe is that all the disks which are part of co-ordinated DG have SCSI3 reservation but my data
disk doesn't have any reservation. My apprehension is that if VCS nodes doesnt do reservation how VCS is
going to protect the disk from outside world, in fact I tried to write on my data disk from a linux box which is not
the part of the cluster?

I am wondering if I have done the configuration correct?

Please find some of command output below which might help to analyze if there is some
configuration mistake:
===============
bash-3.00# vxfenadm -d

I/O Fencing Cluster Information:
================================

 Fencing Protocol Version: 201
 Fencing Mode: SCSI3
 Fencing SCSI3 Disk Policy: raw
 Cluster Members:

          0 (Solaris146)
        * 1 (Solaris148)

 RFSM State Information:
        node   0 in state  8 (running)
        node   1 in state  8 (running)

bash-3.00# hagrp -state
#Group       Attribute             System     Value
groupw       State                 Solaris146 |OFFLINE|
groupw       State                 Solaris148 |ONLINE|
bash-3.00# hares  -state
#Resource    Attribute           System     Value
fodg         State               Solaris146 OFFLINE
fodg         State               Solaris148 ONLINE
fofs         State               Solaris146 OFFLINE
fofs         State               Solaris148 ONLINE
bash-3.00#

============

 
business continuity
Cluster Server
Failover
High availability
Service group
Solaris
  • Gaurav_S's avatar
    Gaurav_S
    15 years ago

    Hi Ajit,

    To answer your questions......

    Once the DG is imported by VCS, you will see the registraiton & reservation of only 1 node (in case of a failover diskgroup) ... However if its a Shared diskgroup (CVM diskgroup), then you would see registration & reservation of both nodes....

    Yes you are right, VCS diskgroup agent imports the diskgroup with specific options ( -o groupreserve) then only reservations are placed.... If you have a normal filesystem where diskgroup was not imported with above options, then it is NOT protected by VCS IOFencing..... however you can configure that FS in VCS ... so basically you just loose the split brain protection (which is real purpose of fencing)....

    Gaurav

7 Replies

Sort By
  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    15 years ago

    Hello Ajit,

    In Veritas IOFencing, coordinator disks get only registration keys however data disks will have both registration keys & reservation keys...

    To check Registration keys..

    # /sbin/vxfenadm -g  /dev/rdsk/<cxtxdx>

    To check reservation keys..

    # /sbin/vxfenadm -r  /dev/rdsk/<cxtxdx>

    Since you have configured VCS as well, did the diskgroups were imported by VCS ?  because VCS will import the diskgroups with reservations... then only you will see reservation keys...

    Gaurav
  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    15 years ago

    Also, just running of vxfen module doesn't indicate that fencing is active....

    Does your main.cf had an entry "UseFence = SCSI3" ? If yes, then only cluster would be using IOfencing...

    As you mentioned you tried writing via a linux box .... how did it go ? did it succeed ? Only that node can write to the disk who has registration keys & wrtie-exclusive reservation key ... your linux box should get an error while attempting write...

    Gaurav
  • ajit_jain's avatar
    ajit_jain
    Level 2
    15 years ago
    Hi Gaurav,

    Thanks for your quick reply.

    As you said I could see the registration as well as reservation once VCS imported the disk group.
    But at a time I only see one registration (from the node where DG is currently online) on the disk while I was expecting two registration keys as it is two node cluster.

    One more question as we started seeing SCSI keys after VCS imported VxVM diskgroup does that means if I were to create a VCS HA file-system on a raw disk then the disk would not be protected by VCS?

    thanks again,
    ajit jain
  • ajit_jain's avatar
    ajit_jain
    Level 2
    15 years ago
    In reply to Gaurav's previous message:-

    Yes, in main.cf I have added entry "UseFence = SCSI3", I believe VCS will not start if there is discrepency in main.cf and vxfen.
    From linux box it succeeded at the first time because VCS has not imported the diskgroup but later it failed.
    But I dont see any reservation or registration for the raw disk (without VXVM diskgroup) which are part of a hares of Mount type.

    thanks,
    ajit
  • Dev_Roy's avatar
    Dev_Roy
    Level 6
    15 years ago

    Ajit,

    The essential components of VCS I/O fencing are as follows:

    1.> LLT, GAB, and VXFEN drivers
    2.> Coordinator disks with SCSI-3 Persistent Reservations (PR)
    3.> Veritas Volume Manager (VxVM) and Dynamic Multipathing (DMP)
    4.> VCS I/O fencing data disks using CVM/DMP and VCS Disk Group agent

    As you could see in point number 3 that volume manager is required, that means that it assumes that disks / luns / arrays are being managed by volume manager. For any other disk (without VXVM diskgroup), you would not see fencing keys in it.

    Hope this helps.

    Regards,
    Dev


  • Gaurav_S's avatar
    Gaurav_S
    Moderator
    15 years ago

    Hi Ajit,

    To answer your questions......

    Once the DG is imported by VCS, you will see the registraiton & reservation of only 1 node (in case of a failover diskgroup) ... However if its a Shared diskgroup (CVM diskgroup), then you would see registration & reservation of both nodes....

    Yes you are right, VCS diskgroup agent imports the diskgroup with specific options ( -o groupreserve) then only reservations are placed.... If you have a normal filesystem where diskgroup was not imported with above options, then it is NOT protected by VCS IOFencing..... however you can configure that FS in VCS ... so basically you just loose the split brain protection (which is real purpose of fencing)....

    Gaurav
  • ajit_jain's avatar
    ajit_jain
    Level 2
    15 years ago
    Hi Dev/Gaurav,
    All your messages really helped.


    thanks a lot.
    ajit jain