Vers Date Who Description ---- ---- --- ----------- v0.01 31-Mar-2016 sdo Several hours of initial preparation. v0.02 6-Apr-2016 sdo First near complete version. v0.03 7-Apr-2016 sdo Updated to use cover using a 512MB vRAM OS install. v0.04 8-Apr-2016 sdo Tweaks to text during a test run. v0.05 10-Apr-2016 sdo Confirm vRAM requirements. Reduce system volume of mhvtl server to 5GB. v0.06 11-Apr-2016 sdo Added basic steps for iSCSI initiator for Windows 2008 R2 SP1. Introduction ------------ This document describes the ten relatively easy steps to provision an instance of an iSCSI based mhvtl server. This is followed by a further five steps which you can follow if you want to onboard the mhvtl "tape" functionality into NetBackup. The associated script automates the 30 or so otherwise tedious steps in provisioning a mixed media "mhvtl" instance of one virtual tape library, plus four virtual tape drives, plus 38 virtual media each of 1GB. The four virtual tape drives are implemented as two LTO4 plus two LTO5 virtual tape drives, and the virtual media are in two sets, one of 19 virtual LTO4 media, and another of 19 virtual LTO5 media. Please take a few moments to read the *entire* header section of the associated script. Whilst the text below may appear to be the full story, it is highly recommended that you also read the entire header section of the script - which contains a fair amount of useful information as to the purpose and intentions of the script itself. The text below is purely related to the few "required" manual steps surrounding provisioning an instance of mhvtl on a CentOS minimal VM. Whereas the header of the script covers some detail around the workings of the script itself. Please do read both. It will only take a few minutes, and you will have a much better understanding of the entire process if you do read them both. Environment ----------- This procedure, and the accompanying script, were developed and tested using: VM host HW: MacBook Pro (13-inch, Late 2011) (MacBookPro8,1) VM host OS: MacOSX El Capitan 10.11.4 VM host hypervisor: VMware Fusion Pro 8.1.0 guest VM VTL HW: 1 x vCPU, 512MB vRAM, 1 x 5GB vHDD, 1 x 40GB vHDD guest VM VTL OS: CentOS 6.7 minimal guest VM VTL app: mhvtl 1.5.4 guest VM NetBackup HW: 2 x vCPU, 2GB vRAM, 1 x 40GB vHDD, 1 x 40GB vHDD guest VM NetBackup OS: Windows Server 2008 R2 SP1 guest VM NetBackup app: NetBackup Server v7.6.1.2 (installed as Master, and then configured as Master/Media) guest VM NetBackup iSCSI: Microsoft iSCSI initiator LAN for general use: 10.0.1.0/24 LAN for iSCSI and backups: 10.0.9.0/24 Background ---------- The mhvtl software set implements a software application which some might otherwise be inclined to consider as being a "VTL" i.e. a Virtual Tape Library. In fact, mhvtl is more than this. And instead, mhvtl should be considered as a tape library "simulator", because it implements features not normally found on commercial VTL software, such as SCSI based Tape Alerts and cleaning tape media. Ordinarily one might consider using mhvtl alongside an installation of backup software such that both the backup software and mhvtl are installed and present within the same OS instance, simply because mhvtl implements psuedo SCSI devices which all appear to the locally installed backup software as being traditional SCSI devices which have previously been presented to the same local host. However, this script extends that functionality such that the SCSI devices (which are mhvtl tape library robot and tape library psuedo devices) are re-presented out via iSCSI. The intention of this script is not to have any backup product installed alongside mhvtl. Instead, the intention of this script is to quickly install and configure a standalone (probably virtual) instance of mhvtl within a server OS instance which itself essentially presents out nothing but iSCSI devices... so that other external OS instances which do host the backup software can access the iSCSI devices which in turn resolve to the psuedo SCSI devices of mhvtl. Also note that another developer has previously created a management GUI for mhvtl. This was not developed by Mark, but was developed by 'nia', and is not part of the standard base mhvtl development. 'nia' has since ceased developing and maintaining this GUI, but Mark has taken over hosting and management of the code base for this GUI, which has a GIT repository name of 'mhvtl-gui'. FYI - CentOS 6.7 has a published EOL of 30th November 2020 - so, as of April 2016, about 4.5 years left. Links ----- You can follow Mark's terrific work on mhvtl here: https://sites.google.com/site/linuxvtl2/ ...and join in the fun here: http://mhvtl-a-linux-virtual-tape-library.966029.n3.nabble.com The script which accompanies this process always pulls down (via GIT) the latest version of mhvtl that Mark may have published. Out Of Scope ------------ This procedure does not cover how, in any detail whatsoever, to install or setup or configure a test NetBackup environment. Such topics have been covered many times in the Veritas NetBackup community forums. This procedure, and the associated script, deal purely with quickly provisioning an instance of mhvtl which can be accessed and utilised via iSCSI. The mhvtl instance which is provisioned by this script should be usable by any of the main enterprise backup software products, and as far as I know, there is nothing inherently private to NetBackup about the mhvtl software itself. This document does have some useful tips at the end which are specific to NetBackup, but apart from that... I believe this procedure and script could be used to create a standalone mhvtl iSCSI based server for pretty much any backup product. Networking ---------- On my test hypervisor (i.e. VMware Fusion Pro on a Mac) I use two networks: 10.0.1.0/24 for all normal outgoing/internet client traffic 10.0.9.0/24 for all internal "server" and backup traffic And the VM which runs mhvtl has two vNIC configured. The first is eth0 which (in my test environment) resides in subnet 10.0.1.0/24. The second is eth1 which (in my test environment) resides in subnet 10.0.9.0/24. The iSCSI devices (which in turn point to the mhvtl devices) are not presented out to the internet or the local LAN via eth0, and are instead presented out only to the backup testing subnet of 10.0.9.0/24 via eth1. And the sshd service/daemon is also configured to only listen on the subnet of eth1. The result of this should be a semi-secure virtual machine which is not listenning on the shared LAN, i.e. it remains unreachable for iSCSI and ssh traffic from the main local LAN and internet. It is typically best practice to use a fixed IP address for both iSCSI target servers, and also for backup servers, even in test environments. This procedure leaves eth0 in DHCP mode. And, whilst the script does not check for a fixed IP address configuration on eth1, what may happen is that... if the mhvtl VM is rebooted after a few hours or days of usage, then the previously acquired DHCP address on eth1 could change, and thus the mhvtl instance would longer be accessible from the backup host. Hence it is good idea to use a fixed IP for interfaces on servers which host "server" side services. This script will still work even if only one NIC (or vNIC) is present, if you just want to keep networking simple. Even still, it is probably a good idea to then configure a fixed IP address on eth0 (i.e. if you do decide to implement an OS instance with just one NIC/vNIC). Before Starting --------------- The most important first step is to consider your networking, and determine a fixed address that will be used on eth1 for iSCSI traffic on your VTL - i.e. before starting, determine the fixed IP address that you will use on eth1 (if you have your VTL VM in a dual virtual network environment), or on eth0 (if you have your VM in a single network). Also, before you start, decide upon a host name for your mhvtl server - choose carefully, as the server's hostname will be used to form part of the iSCSI target IQN name. If you've never used the Linux/Unix "vi" editor before, then you might want to read up on the basics first. If you don't prepare yourself for the oddities of "vi", then you could get in a bit of a mess. You may want to use the "nano" editor instead of "vi", in which case change any instruction below from using "vi" to use "nano". How Much vRAM For The VM Which Will Be Running mhvtl? ----------------------------------------------------- An instance of mhvtl with one library and four tape drives should run fine in 512MB of vRAM with no swaping occuring, in fact there should be about 115MB of vRAM free (for a 512MB VM) when all four tape drives are busy writing. VMware Fusion Pro 8.1.0 offers a recommended vRAM of 1GB for a CentOS VM. When you choose 512MB for your VM then the process below should work just fine... BUT... if you chose 1GB or more vRAM for your VM, then the initial OS installation pathway is different, and you end up going through a hi-res GUI based installation process rather than a text based GUI install when you choose 512MB of vRAM. Fair enough, but the issue with following a 512MB (or less) text based install is that it leaves you with a VM that has no host name, and also has no default networking configured. Anyway, the procedure below caters for both circumstances. Why Reboot Halfway? ------------------- The mhvtl application uses some kernel mode drivers. And because these need to be compiled against the kernel of the running OS instance, therefore we need to reboot so that the latest updated kernel is fully loaded so that mhvtl can be compiled against the installed and running OS kernel version. OS Disk Sizing -------------- Through the stages the used/free space of the 5GB primary volume (sda) was tracked as: After installing base OS, before script 1st run Size 3.9 GB 616 MB used 3.1 GB free 17% used After script 1st run, before script 2nd run Size 3.9 GB 1000 MB used 2.7 GB free 27% used After script 2nd, i.e. downlaod source, compile Size 3.9 GB 1.1 GB used 2.6 GB free 30% used mhvtl Sizing ------------ This procedure, i.e. the script, will create a VTL with a capacity of 38 writeable media, each of 1000 MB capacity, i.e. a total of 37.1GB. Hence the VM has a second hard disk drive of 40GB. Whilst it is likely that any backup data will compress, and so the VTL may well be able to ingest and store more than 37.1GB of backup data, we need to be mindful that the backup data may already be compressed, i.e. that the virtual media all have the capability of storing 1000MB each, or that we may be using encryption, and so we only need to be be able to store 37.1GB of "post tape processing" data which may well be more than 37.1GB of ingested backup data... and we also need to allow a little bit of extra room for the mhvtl virtual media "indx" and "meta" control files, hence a second VM volume total size of 40GB. Procedure --------- N.B: Remember to change any use of IP address 10.0.9.242, in the example steps below, to your chosen IP address for eth1 (in a dual network implementation), or eth0 (in a single network implementation), for your build/network/environment. 1) Download Centos 6.7 x86_64 minimal ISO from: http://mirrors.melbourne.co.uk/sites/ftp.centos.org/centos/6.7/isos/x86_64/ 2) Create an empty VM: 1 x vCPU 512MB vRAM 2 x vNIC one in "auto detect", and one in "vmnetN" 2 x vHDD one at 5 GB for the OS, and one at 40 GB for the VTL tape media ...dis-allow VM hardware upgrades ...dis-allow sharing bluetooth ...remove sound card ...remove printer ...remove camera ...enable "connect" CD/DVD drive, and point at the CentOS 6.7 Minimal ISO file ...close the guest VM config dialog box. -) Optional step... -) Take a VM snapshot, and name the snapshot as: "1-before-OS" 3) Install Centos... Pathway 1 - When Your VM Has 512MB of vRAM - Text Based GUI Install ------------------------------------------------------------------- ...boot the VM to begin the install of CentOS 6.7 from the minimal DVD... Welcome to CentOS 6.7! select: Install or upgrade an existing system Disc Found select: Skip Welcome to CentOS select: OK Language Selection select: English select: OK Keyboard Selection select: uk select: OK Warning select: Re-initialise all Time Zone Selection: select: Europe/London select: OK Root Password: enter your password twice select: OK Partitioning Type: select: Use entire drive select: sda de-select: sdb select: OK Writing storage configuration select: Write changes to disk ...and the installation will begin, and after a few minutes... select: Reboot Pathway 2 - When Your VM Has 1GB or More of vRAM - Hi-Res GUI Based Install --------------------------------------------------------------------------- ...boot the VM to begin the install of CentOS 6.7 from the minimal DVD... ...N.B: we will NOT be configuring networking during the install... so, please, do NOT be tempted to enter that section... (...take to first option...) select "Install or upgrade an existing system" Disc Found select "Skip" CentOS 6 screen: click: Next Language... select: English click: Next Keyboard... select: United Kingdom click: Next What type of devices... select: Basic Storage Devices click: Next Storage Device Warning... click: Yes, discard any data Hostname: enter: vtlNN.localdomain click: Next (N.B: do NOT "Configure Network") Nearest city: select: a city click: Next Root account password: click: Next What type of installation: select: Use All Space click: Next Data storage devices: select: the 5??? MB disk... click: -> arrow click: Next click: Write changes to disk ...and the installation will begin, and after a few minutes... click: Reboot -) Optional step: -) Logon at the console as root, and shutdown the VM using: shutdown -h now -) Take a VM snapshot, and name the snapshot as: "2-after-OS-before-network" -) Power-on the VM. 4) Configure networking: ...at the VM's console, login as root... ...list interfaces: ip a ...configure eth0: vi /etc/sysconfig/network-scripts/ifcfg-eth0 ...and change this one line: ONBOOT=yes :wq! ...configure eth1: vi /etc/sysconfig/network-scripts/ifcfg-eth1 ...and change these two lines: ONBOOT=yes BOOTPROTO=none ...and add these lines to the end: PEERDNS=no IPADDR=10.0.9.242 :wq! ...and then start networking... ...if you used 512MB vRAM then networking will not have been previously started by the text based GUI installer... ...if you used 1GB vRAM then networking will have been previously started by the hi-res based GUI installer... service network status service network stop service network start service network status ...relist the interfaces: ip a ...but don't worry if "eth1" initially has the wrong address at this very moment, this will be resolved by the reboot just below... ...and check DNS which should now have been populated with DNS servers detected via DHCP via the eth0 subnet: cat /etc/resolv.conf 5) 5) Check/configure the hostname... ...if you used 512MB vRAM and followed the text GUI installer... then you will need to set the hostname manually... ...if you used 1GB vRAM and followed the hi-res GUI installer... then you should not need to set the hostname manually... vi /etc/sysconfig/network # ...and change the HOSTNAME= entry, to something like: HOSTNAME=xxxxxxxx.localdomain # ...where xxxxxxxx is the hostname for your mhvtl VM... # ...perhaps something like: HOSTNAME=vtl01.localdomain # ...and then save your changes using: :wq! 6) Create a folder to receive the build/configuration script: mkdir -pv /home/vtladmin ...reboot so that the networking config takes hold: reboot now ...and then minimize the VM console. -) Optional step: -) Logon at the console as root, and shutdown the VM using: shutdown -h now -) Take a VM snapshot, and name the snapshot as: "3-after-network-before-script-1st-run" -) Power-on the VM. -) Minimize the VM console. 7) On the Mac VM hypervisor host (copy the script which accompanies this procedure) to the new CentOS VM... ...start a "Terminal" session on your Mac, and then use: cd ~ cd Downloads scp config-mhvtl-centos67minimal-netbackup.sh root@10.0.9.242:/home/vtladmin/ 8) Now logon to the VM via ssh from Mac hypervisor VM host: ssh root@10.0.9.242 ...and execute the script for the first time: cd /home/vtladmin ls -lash ./config-mhvtl-centos67minimal-netbackup.sh ...the script will run for about three or four minutes, before it needs to reboot... ...but before rebooting, see the next step... 9) Set a password on the "vtladmin" username: passwd vtladmin ...then reboot the VM: reboot now -) Optional step: -) Logon at the console as root, and shutdown the VM using: shutdown -h now -) Take a VM snapshot, and name the snapshot as: "4-after-script-1st-run-before-script-2nd-run" -) Power-on the VM. -) Minimize the VM console. 10) Wait for the VM to reboot, and also remember that this time we will logon (from the Mac "Terminal" session) as user "vtladmin": ssh vtladmin@10.0.9.242 ...then su to root so that we can finish the config: su - ...and now go to the folder containing the script: cd /home/vtladmin ...and re-run the script, which will re-check, and then continue from where it left off: ./config-mhvtl-centos67minimal-netbackup.sh ...and, when you see a summary like: 11/04/2016 15:58:06 Step 30 - Final summary... 11/04/2016 15:58:06 ...list of SCSI devices... [0:0:0:0] disk VMware, VMware Virtual S 1.0 /dev/sda /dev/sg0 [0:0:1:0] disk VMware, VMware Virtual S 1.0 /dev/sdb /dev/sg1 [2:0:0:0] cd/dvd NECVMWar VMware IDE CDR10 1.00 /dev/sr0 /dev/sg2 [3:0:0:0] mediumx STK L700 0105 /dev/sch0 /dev/sg7 [3:0:1:0] tape IBM ULT3580-TD4 0105 /dev/st0 /dev/sg3 [3:0:2:0] tape IBM ULT3580-TD4 0105 /dev/st1 /dev/sg4 [3:0:3:0] tape IBM ULT3580-TD5 0105 /dev/st2 /dev/sg5 [3:0:4:0] tape IBM ULT3580-TD5 0105 /dev/st3 /dev/sg6 11/04/2016 15:58:06 ...versions used and installed... 11/04/2016 15:58:06 ...script version: v0.23 11/04/2016 15:58:06 ...OS version: 2.6.32-573.22.1.el6.x86_64 11/04/2016 15:58:06 ...mhvtl version: 0.18.17 11/04/2016 15:58:07 ...vtlcmd version: 1.5.4-git-c7ccb7a 11/04/2016 15:58:07 ...tgt version: 1.0.24 11/04/2016 15:58:07 ...iscsi version: 6.2.0-873.13.el6 11/04/2016 15:58:07 ...iscsi listenner: 10.0.9.242 11/04/2016 15:58:07 ...iscsi target: iqn.2016-02.us.mhvtl:tgt:vtl01.localdomain 11/04/2016 15:58:07 ...done... 11/04/2016 15:58:07 11/04/2016 15:58:07 Script completed... 11/04/2016 15:58:07 Script exiting... ...then the preparation of the mhvtl server is complete... ...so now logoff from the VM: exit ...and logoff from the Mac Terminal session: exit -) Optional step: -) Logon at the console as root, and shutdown the VM using: shutdown -h now -) Take a VM snapshot, and name the snapshot as: "5-after-script-2nd-run-before-using" -) Power-on the VM. -) Minimize the VM console. 11) It is assumed that you already have another server OS instance somewhere... ...upon which you have already installed NetBackup, and so... ...on your instance of NetBackup Server (master, master/media, media) connect via iSCSI... ...to the IP address of 10.0.9.242 of the "mhvtl" server that we have just created, and... ...discover/configure the tape library and tape drive iSCSI devices. For example: On a Windows 2008 R2 SP1 server, two update actions have to be taken, which are embedded within the 8 steps below, as follows: 1) Open Control Panel 2) Open iSCSI Initiator 3) Click on the "Targets" tab 4) Enter the IP address of the mhvtl server, in this example: 10.0.9.242 ...and the "Discovered targets" box, just below should now show a "Status" of "connected" 5) Click on the "Volumes and Devices" tab 6) Click the "Auto Configure" button ...and the "Volume List" panel above should populate with four devices. 7) Clck OK 8) Close Control Panel ...and to confirm device visibility by NetBackup, you could run these NetBackup Volume Manager commands: scan -changer scan -tape ...and then reboot the iSCSI initiator client, i.e. your NetBackup (Master/)Media Server. 12) Now run the "Device Discovery Wizard" on the NetBackup Master Server. 13) Configure NetBackup Volume Manager to use the "mhvtl" media: vmpool -listall -b vmpool -list_scratch vmpool -add ScratchPool "Scratch pool" ANYHOST -1 -2 vmpool -set_scratch ScratchPool vmpool -listall -b vmquery -list_media_genrule vmquery -add_media_genrule 0 8 "1:2:3:4:5:6" vmquery -list_media_genrule vmrule -listall -b vmrule -verbose -add "4" hcart ScratchPool 0 "LTO4 media" vmrule -verbose -add "5" hcart2 ScratchPool 0 "LTO5 media" vmrule -verbose -add "CLN4" hcart_clean None 25 "LTO4 cleaner" vmrule -verbose -add "CLN5" hcart2_clean None 25 "LTO5 cleaner" vmrule -listall -b # ...inventory the robot and list media present in robot: vmquery -b -rn 0 vmupdate -rn 0 -rt tld -use_barcode_rules -use_seed vmquery -b -rn 0 14) If this is the first time you have inventoried your VTL, i.e. if you haven't used any of the virtual media before in NetBackup, then you shouldn't need to label the media. However, if you have used the same media IDs in your NetBackup instance before, e.g. you re-created the mhvtl VM, then you will need to remember to relabel all of the virtual media before you can write to them again - because NetBackup remembers if it can expect to be able to identify the media label, i.e. NetBackup records (in its database) whether the media has been written to before or not - so, if you create your mhvtl instance after having previously used the same media IDs then NetBackup will freeze the new media because it won't be able to find a media label when it expects to - hence you will need to re-label any re-created media, i.e. any re-used media IDs. FYI - there are two NetBackup KMS configuration steps below... ...the first is for a Linux/Unix based NetBackup Master Server... ...the second is for a Windows based NetBackup Master Server... 15a) Configure NetBackup KMS encryption on a Linux/Unix based NetBackup Master Server: # ...create the volume pool: vmpool -listall -b vmpool -create -pn ENCR_POOL -description "KMS encrypted" vmpool -listall -b # ...create the key database: nbkms -info nbkms -createemptydb passphrase my-hmk-id passphrase my-kpk-id nbkms -info # ...has the NetBackup KMS daemon started: ps -ef | grep -i nbkms # ...if not then start it with... # ...use NetBackup Java Admin Console, connect to master, the browse to Activity Monitor, and Daemons tab... # ...and start the 'nbkms' daemon. # ...list the newly created details: nbkmsutil -gethmkid nbkmsutil -getkpkid ls -lash /usr/openv/kms/db ls -lash /usr/openv/kms/key # ...create the key group - N.B. the keygroup name MUST begin with the five character string of 'ENCR_' nbkmsutil -listkgs nbkmsutil -createkg -kgname ENCR_POOL nbkmsutil -listkgs # ...create the key: nbkmsutil -listkeys -kgname ENCR_POOL nbkmsutil -createkey -keyname my-key -kgname ENCR_POOL # ...the above will prompt for a key pass phrase... nbkmsutil -listkeys -kgname ENCR_POOL nbkmsutil -modifykey -keyname my-key -kgname ENCR_POOL -activate nbkmsutil -listkeys -kgname ENCR_POOL nbkmsutil -ksstats # ...in use... # Configure backup/SLP policy to use the new Volume group (i.e. it must begin with "ENCR_"). # Do not enable "encryption" on policy - as this is for client side encryption. # Run a test backup. # Confirm encryption, using images on media report. # ...below are the minimum steps to backup the NetBackup KMS key database: # ...ideally amend this to copy off-host and off-site... # ...when NetBackup KMS passphrases based keys are used... # ...then this only needs to be run when keys are added, changed, amended... # ...and does not need to be run after each backup session. nbkmsutil -ksstats nbkmsutil -quiescedb nbkmsutil -ksstats cp -pv /usr/openv/kms/db/KMS_DATA.dat /somewhere cp -pv /usr/openv/kms/key/KMS_HMKF.dat /somewhere cp -pv /usr/openv/kms/key/KMS_KPKF.dat /somewhere nbkmsutil -ksstats nbkmsutil -unquiescedb nbkmsutil -ksstats exit 15b) Configure NetBackup KMS encryption on a Windows based NetBackup Master Server... REM ...create the volume pool: vmpool -listall -b vmpool -create -pn ENCR_POOL -description "KMS encrypted" vmpool -listall -b REM ...create the key database: nbkms -info nbkms -createemptydb passphrase my-hmk-id passphrase my-kpk-id nbkms -info REM ...start the NetBackup KMS Service: net start | find "Key" net start "NetBackup Key Management Service" net start | find "Key" REM ...list the newly cerated details: nbkmsutil -gethmkid nbkmsutil -getkpkid dir /s /b "D:\Program Files\Veritas\kms\" REM ...ensure service starts with NetBackup, i.e. set to Automatic: services.msc REM ...create the key group - N.B. the keygroup name MUST begin with the five character string of 'ENCR_' nbkmsutil -listkgs nbkmsutil -createkg -kgname ENCR_POOL nbkmsutil -listkgs REM ...create the key: nbkmsutil -listkeys -kgname ENCR_POOL nbkmsutil -createkey -keyname my-key -kgname ENCR_POOL REM ...the above will prompt for a key pass phrase... nbkmsutil -listkeys -kgname ENCR_POOL nbkmsutil -modifykey -keyname my-key -kgname ENCR_POOL -activate nbkmsutil -listkeys -kgname ENCR_POOL nbkmsutil -ksstats REM ...in use... REM Configure backup/SLP policy to use the new Volume group (i.e. it must begin with "ENCR_"). REM Do not enable "encryption" on policy - as this is for client side encryption. REM Run a test backup. REM Confirm encryption, using images on media report. REM ...below is a simple script to backup the NetBackup KMS key database: REM ...ideally amend this to copy off-host and off-site... REM ...when NetBackup KMS passphrases based keys are used... REM ...then this only needs to be run when keys are added, changed, amended... REM ...and does not need to be run after each backup session... REM ...remember to adjust the drive volume letters according to your Windows based NetBackup Master Server install... @echo on setlocal enabledelayedexpansion nbkmsutil -ksstats nbkmsutil -quiescedb nbkmsutil -ksstats dir /s /b "D:\Program Files\Veritas\kms\" copy "D:\Program Files\Veritas\kms\db\KMS_DATA.dat" "J:\NBU_KMS_BACKUP\KMS_DATA.dat" copy "D:\Program Files\Veritas\kms\key\KMS_HMKF.dat" "J:\NBU_KMS_BACKUP\KMS_HMKF.dat" copy "D:\Program Files\Veritas\kms\key\KMS_KPKF.dat" "J:\NBU_KMS_BACKUP\KMS_KPKF.dat" nbkmsutil -ksstats nbkmsutil -unquiescedb nbkmsutil -ksstats pause exit /b REM to restore... copy "J:\NBU_KMS_RESTORE\KMS_DATA.dat" "D:\Program Files\Veritas\kms\db\KMS_DATA.dat" copy "J:\NBU_KMS_RESTORE\KMS_HMKF.dat" "D:\Program Files\Veritas\kms\key\KMS_HMKF.dat" copy "J:\NBU_KMS_RESTORE\KMS_KPKF.dat" "D:\Program Files\Veritas\kms\key\KMS_KPKF.dat" [end]