Making an existing volume site consistent

[joagmv]

Hello,

We have a dg which should be site consistent but it is not. Setting up the dg as site consistent (disk tagging, sites added and siteconsistent flag enabled), how to make a volume site consistent?

In the storage foundation cfs and high avalaibility the page 539 says:

   8 Turn on the allsites flag for the volume which requires data replication to each site:

   # vxvol [-g diskgroup] set allsites=on volume

   9 Turn on site consistency for each existing volume in the disk group for which

   siteconsistency is needed. You also need to attach DCOv20 if it is not attached

   already. DCOv20 is required to ensure that site detach and reattach are

   instantaneous.

   #  vxvol [-g diskgroup] set siteconsistent=on volume ...

But I think it is not correct as you first should mirror the volume and create the dco log:

   # vxvol -g test_dg set allsites=on no_mirror1_lv

   VxVM vxvol ERROR V-5-1-12891  No plex found for vol no_mirror1_lv on site Y

   VxVM vxvol ERROR V-5-1-12889  Cannot enable allsites on volume no_mirror1_lv

Am I wrong?? What I do is as follows:

1. Make sure the dg is site consistent

2. Create a mirror (plex in the other site) and a dco log:

     # vxassist -g test_dg mirror no_mirror1_lv mirror=site

     # vxsnap -g test_dg prepare no_mirror1_lv

3. Check that plexes have been created as siteconsist (there is one in each site)

4. Enable flags and set up the policy read:

     # vxvol -g test_dg set siteconsistent=on no_mirror1_lv

     # vxvol -g test_dg set allsites=on no_mirror1_lv

     # vxvol -g test_dg rdpol siteread no_mirror1_lv

# vxprint -g test_dg -m no_mirror1_lv  | grep -e fastresync -e dco_name -e siteconsistent -e allsites

fastresync=on

dco_name=no_mirror1_lv_dco

siteconsistent=on

allsites=on

v  no_mirror1_lv -           ENABLED  ACTIVE   20971520 SITEREAD  -        fsgen

pl no_mirror1_lv-01 no_mirror1_lv ENABLED ACTIVE 20971520 CONCAT  -        RW

sd ibm_ds8x000_9c00-01 no_mirror1_lv-01 ibm_ds8x000_9c00 0 20971520 0 ibm_ds8x000_9c00 ENA

pl no_mirror1_lv-02 no_mirror1_lv ENABLED ACTIVE 20971520 CONCAT  -        RW

sd ibm_ds8x001_9c00-01 no_mirror1_lv-02 ibm_ds8x001_9c00 0 20971520 0 ibm_ds8x001_9c00 ENA

dc no_mirror1_lv_dco no_mirror1_lv no_mirror1_lv_dcl

v  no_mirror1_lv_dcl -       ENABLED  ACTIVE   67840    SITEREAD  -        gen

pl no_mirror1_lv_dcl-01 no_mirror1_lv_dcl ENABLED ACTIVE 67840 CONCAT -    RW

sd ibm_ds8x000_9c00-02 no_mirror1_lv_dcl-01 ibm_ds8x000_9c00 20971520 67840 0 ibm_ds8x000_9c00 ENA

pl no_mirror1_lv_dcl-02 no_mirror1_lv_dcl ENABLED ACTIVE 67840 CONCAT -    RW

sd ibm_ds8x001_9c00-02 no_mirror1_lv_dcl-02 ibm_ds8x001_9c00 20971520 67840 0 ibm_ds8x001_9c00 ENA

------------------

- Are these steps correct? are there these steps in the official documents or any guide?

- Is it mandatory to enable the flag allsites??? what does it do?? (It seems to me that it is useful before the volume creation).

Regards,

joagmv

[mikebounds · Accepted solution]

I find there is a lot of confusion about what site consistency provides, so this is my understanding:

Suppose you have 2 volumes, with a single Oracle Database on and both volumes are mirrored across sites , Oracle is running at site1 and site1 looses connection to just ONE of the plexes at site2.

WITHOUT site consistency - this is what would happen:

The plex at site2 is detached and Oracle on site1 continues to write data to both plexes at site1, but only one plex at site2.  Site1 fails, so you then switch to site2, and site2 can see the local plex for each volume, but the data is useless as one volume has up-to-date data and the other does not.

WITH site consistency - this is what would happen:

ALL plexes at site2 which have siteconsistent=on are detached and Oracle on site1 continues to write data to both plexes at site1, but the one visable plex at site2 IS not written to as all plexes were detached.  Site1 fails, so you then switch to site2, and site2 can see the local plex for each volume, and although the data is old, it is consistent so it can be used.

So if you are configuring site consistency, the assumption is that your volumes are already mirrored and configuring site consistency just ensures a site remains consistent so that some plexes don't have newer data then others on the same site.

Setting allsites=on for a volume means if you try to add a new site to the diskgroup, this will only work if the volumes with allsites=on have a plex on site that is being added.

Mike

[Douglas_Snyder]

Hi,

The guide for setting up site aware storage is in the Cluter File System Administrator's Guide for your OS (Linux is below).

http://www.symantec.com/business/support/resources/sites/BUSINESS/content/live/DOCUMENTATION/5000/DO...

Please refer to page 532 for the setup and configuration.

Doug Snyder, CISSP

All documents - all products, all OS types:

http://sfdoccentral.symantec.com/

[joagmv]

OK, I see now the meaning of site consistent and allsites. So, the steps I follow are correct in order to convert a volume into a mirror volume with site consistency, right??

Would you change any step???

Regards,
joagmv

[mikebounds]

There are some steps missing - you mentioned steps 8 and 9 in your opening post, but you don't have all of the other 7 on page 539 (like step 6 for instance):

1 Ensure that the disk group is updated to at least version 140, by running the

vxdg upgrade command on it:

# vxdg upgrade diskgroup

2 On each host that can access the disk group, define the site name:

# vxdctl set site=sitename

3 Tag all the disks in the disk group with the appropriate site name:

# vxdisk [-g diskgroup] settag site=sitename disk1 disk2

Or, to tag all the disks in a specified enclosure, use the following command:

# vxdisk [-g diskgroup] settag site=sitename encl:encl_name

4 Use the vxdg move command to move any unsupported RAID-5 volumes to

 another disk group. Alternatively, use the vxassist convert commands to

convert the volumes to a supported layout such as mirror or mirror-stripe.

You can use the site and mirror=site storage allocation attribute to ensure

that the plexes are created on the correct storage.

5 Use the vxevac command to ensure that the volumes have at least one plex

 at each site. You can use the site and mirror=site storage allocation attribute

to ensure that the plexes are created on the correct storage.

6 Register a site record for each site with the disk group:

# vxdg -g diskgroup addsite sitename

7 Turn on site consistency for the disk group:

# vxdg -g diskgroup set siteconsistent=on

Mike

[joagmv]

I already implemented these steps before starting to create the mirror of the volume although I didn't pasted here.

Doubt resolved! Thanks!

Regards,
joagmv