VCS One- Backup and Recovery

[Kyle_Gleed]

Applicable Releases:  VCS One 2.0 & 2.0RU1
10/5/2009

VCS One Backup and Recovery
After implementing VCS One it’s important to have a good data protection plan to protect your Policy Master.  The following discusses best practices for backing up the VCS One Policy Master.

What to Backup
There are two parts to a successful VCS One backup strategy:  backing up the PM database and backing up the VCS One security brokers.

Backing up the PM Database
There are three methods for backing up the PM database:

1.    Database backup
2.    XML export
3.    Command file dump

Database backups
A database backup makes a copy of the PM database by quiescing I/O and copying the database file, transaction log, and the odbc.ini file. 

Advantages: 
•    Quick and allows for fast recovery. 

Disadvantage
•    Database backups cannot be edited offline.
•    Database backups cannot be imported by the VCS One simulator. 

To perform a database backup logon to the node where the PM is running (PM must be ONLINE) and execute the following commands:

# mkdir <backup dir>
# cd <bkup dir>
# /opt/VRTSvcsone/bin/haadmin –backup –db .

XML Exports
An XML backup exports the contents of the database into a series of XML files. 

Advantages
•    XML files can be edited offline.
•    XML files can be imported by the VCS One simulator.
 
Disadvantage
•    Restore requires multiple steps to initialize the DB and import the XML files.

To perform an XML backup logon to the node hosting the PM (PM must be ONLINE) and run the following commands:

# mkdir <backup dir>
# cd <backup dir>
# /opt/VRTSvcsone/bin/haconf –dbtoxml .

Command File Exports
A command file backup exports the contents of the database into a series of “ha*” commands that can be used to rebuild the VCS One Cluster configuration. 

Advantages
•    Command files can be edited offline.
•    Command file exports serve as a nice reference for determining command line syntax when writing shell custom scripts used to automate Service Group and Resource provisioning. 

Disadvantages
•    Restoring/rebuilding the VCS One cluster from a command file can be very slow.  

To perform a command file backup logon to the node hosting the PM (PM must be ONLINE) and run the following commands:

# mkdir <backup dir>
# cd <backup dir>
# /opt/VRTSvcsone/bin/haconf –dbtocmd .

Which backup option should you use?
At a minimum, your backup strategy should include daily DB backups together with daily XML exports.  Command file backups are optional based on your needs.  It is recommended to implement a shell script on each node in the PM cluster that is invoked from the crontab.  The script should check if the PM is running on the system and if so perform the backup.  If the PM is not running it should take no action and exit. 


Backing up the AT Security Brokers
Security in a VCS One cluster is provided by the Symantec Product Authentication Services (AT).  AT is comprised of a hierarchy of brokers that deploy a PKI-like architecture to encrypt communication and perform client and user authentication.

There are two types of brokers in a VCS One Cluster: the Root Broker and the Authentication Broker.

Root Broker (RB)
Each cluster will have one Root Broker (RB).  The RB instantiates the AT security domain and is configured on the first node in the PM cluster. 

If the system hosting the RB fails the RB is lost.  If this happens there is no impact to the running VCS One Cluster, however you will not be able to perform maintenance on the PM cluster (i.e. add/replace a node to the PM cluster) until the RB has been recovered.  As such it’s important to have regular backups of the RB.

Authentication Broker (AB)
Each node in the PM cluster is configured as an Authentication Broker (AB) (note that one node in the PM cluster will host both an RB and AB while the other nodes will host an AB only).  ABs register with the RB and are delegated authority for issuing certificates and performing authentication in the VCS One cluster. 

It is important to perform regular backups of the ABs to protect against server failures in the PM cluster.

Performing AT Broker Backups
Broker’s require files located on both local disk and shared storage.  For this reason a two step backup approach is recommended: 

1.    Run local backups on each node in the PM cluster to protect the local files.

2.    Run a separate backup on the node actively running the PM to protect the shared files.

Local backups
On each node in the PM cluster you need to run a local backup to capture the local broker configuration.  This is done using the following command:

# /opt/VRTSat/bin/vssat showbackuplist

This command backs up the local broker(s) by creating a recovery point in a directory named /var/VRTSatSnapShot where the broker configuration files are copied. 

Notes:
•    AT brokers have hostname dependencies and you can only restore an AT backup to the same system from which it was backed up.  If you rebuild a system you should use the same hostname and IP for the replacement system.
•    Keep track of the system hosting the Root Broker as only that system’s backup will include the RB files.
•    Each successive backup overwrites the previous contents of the /var/VRTSatSnapShot directory, so you need to save a copy of this directory before running the next backup.  

Shared storage backups
Shared files reside on shared storage and can move between systems in the PM cluster.  To ensure that you always have a readily accessible backup of the shared files VCS One provides the following command:

# /opt/VRTSvcsone/bin/haadmin –backup –vss .

This command is a wrapper around the “vssat showbackuplist” command but provides an advantage as it always runs on the PM node that has access to the shared storage. 

Notes:
•    This command automatically creates a tarball of the /var/VRTSatSnapShot directory and puts it in the local directory in which the command was executed.
•    This command can only be executed on the system actively hosting the PM.

Summary of the AT Backup Steps
A two-step backup strategy is recommended to protect the AT brokers in your VCS One cluster:

1.    Implement custom shell scripts on each node in the PM cluster to invoke the ‘/opt/VRTSat/bin/vssat showbackuplist’ command and create a tarball of the /var/VRTSatSnapShot directory.  The tarball name should include the server name, date, and time of the backup.  Schedule the shell script to run once every 24 hours.  This backup script ensures that the local configuration files for the RB and all ABs are backed up each day.

2.    Implement a second shell script that calls the ‘/opt/VRTSvcsone/bin/haadmin –backup –vss .’ command.  Run this script on each node in the PM cluster once each day.  The shell script should check if the PM is active on the local node and if so perform the backup.  If the PM is not active the script should take no action.  This backup will ensure that the shared files (ClusterPDR) are backed up.