02-11-2013 05:22 PM
Hi,
First time performing full Catalog Recovery on Windows Netbackup 7.5 got into an odd issue. Built a new Windows 2003 x64 server in the lab with same SP as in production and installed NetBAckup 7.5.0.1 as in production, difference is in the tape hardware, single T10k, versus a silo in production.
Made a full Catalog backup with a DR file. Load the tape into the drive, added the tape to the CatalogBackup volume pool and successfully completed the Full Catalog Recovery from the Admin console GUI.
As a result – can see all the policies and looks like all the images were restored to …\NetBackup\db\images folder, but nothing else has been restored – NO storage devices, NO media, NO media servers …
So, the question is – what is changed in 7.5 – Do I have to perform any additional steps to recover EMM and NBDB to get the the same configuration as at the production site.
Anybody with NetBackup 7.5 Catalog Recovery ?
Thanks
Solved! Go to Solution.
02-13-2013 07:17 AM
Great idea. The Catalog tapes had the 2 stages backed up, but the tapes were owned by another media server, so even I changed the hostname in the DR file - did not help, but after changing the Media Host Override to the DR media server - the Recovery completed successfully with EMM and NBDB recovered.
Thank you all for the help
02-11-2013 06:17 PM
02-11-2013 06:55 PM
I can confirm that hostname is the same, same patch 7.5.0,1, same installation path d:\veritas
attached is the recovery log.
Thanks Marianne for a fast response.
02-11-2013 07:27 PM
The log file contains no record of Relational database restore.
Did you use the Catalog Recovery wizard in the GUI or cmd (bprecover -wizard) ?
Which option did you choose when you were presented with recovery options?
02-12-2013 06:45 AM
Used wizard in the GUI with Recover the entire catalog. Trying another Catalog tape now.
02-12-2013 06:52 AM
here is the message from the new tape recovery, that shows why the NBDB was not restored. Am I missing something in the Catalog backups?
08:09:40 (9.xxx) WARNING: The following files and directories will not
08:09:40 (9.xxx) be restored because they are not present when the backup
08:09:40 (9.xxx) was done on 2/11/2013 7:03:29 AM. These files and
08:09:40 (9.xxx) directories were either moved or deleted prior to this
08:09:40 (9.xxx) backup, but did exist in a previous backup. If needed,
08:09:40 (9.xxx) they can be restored by doing a normal restore instead
08:09:40 (9.xxx) of doing a true image restore.
08:09:40 (9.xxx) -----------------------------------------------------------
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMRDB.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMRDB.log.1 is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMR_DATA.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/BMR_INDEX.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DARS_DATA.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DARS_INDEX.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DBM_DATA.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/DBM_INDEX.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/EMM_DATA.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/EMM_INDEX.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/JOBD_DATA.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBAZDB.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBAZDB.log.1 is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBDB.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/NBDB.log.1 is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/SEARCH_DATA.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/SEARCH_INDEX.db is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/databases.conf is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/server.conf is not in the true image list. Skipping.
08:09:40 (9.xxx) /D/Veritas/NetBackupDB/staging/vxdbms.conf is not in the true image list. Skipping.
02-12-2013 06:55 AM
Have you checked status of backup done on 2/11/2013 7:03:29 AM?
How many jobs in this backup?
Status of each job?
02-12-2013 07:20 AM
4 Catalog jobs including the parent job. All 4 jobs completed successfully. In the details it show the staging and validating the images.
02-12-2013 07:21 AM
I am trying to copy the \NetBackupDB\staging folder to the DR master and will try:
bprecover -r -nbdb
02-12-2013 07:59 AM
all files has been disappear from \staging folder
02-12-2013 02:48 PM
On the live side, is there any hint of a failure in one of the catalog jobs for the catalog backup. As you will now, there are 4 jobs, parent, staging, backup of staging and backup of image DB.
I think at the moment, the place to look for this issue is during the catalog backup.
Would also be worth running a cat_convert -dump on the .f file, to see for 100% what was backed up. I am sure you would have noticed a backup failure and mentioned it.
Ideas I have ...
Delete and recreate the catalog backup policy.
Run a new full catalog and see if that recovers.
I can't see that this will make much dofference, but soetimes you only find out things by trying them, and just occassionally, doing something you think has no relevance provides a solution.
Martin
02-12-2013 03:22 PM
On the live side, are there any sybolic links that you are using to relocate the /usr/opevn/db or .../db/data dirs ?
Martin
02-12-2013 08:14 PM
@daza - I am curious to know on which master you are looking for staging folder and why?
staging is a temporary phase during backup and restore. As Martin suggested - perform another backup on Prod master and monitor each stream - check Windows explorer while staging phase is running and also when relational database is backed up.
Ensure that you copy the newest DR-file. If I remember correctly, there is a problem with catalog recovery if newer images are found for relational database upon completion of the images restore phase - or not found at all in images.
Complete backup on Prod master followed by recovery on test master.
One more thing: PLEASE upgrade to 7.5.0.4 ASAP - there were some issues with NBU catalogs that were only resolved in 7.5.0.3.
02-12-2013 11:41 PM
Good points Marianne - if there are no sym links floating around, then my next idea would be some issue with te DR file, or the wrong DR file being used.
THe suggestion of a new full, would mean there is only the refernece to one backup in the DR file, which simplifies things a little.
Martin
02-13-2013 01:27 AM
Just one more thought that has caught people out before ... do you have any exclusions set on your live Master, just in case files are being skipped.
To double check on the live Master open up the BAR GUI select the Master as the source client and NBU-Catalog as the policy type and see if all the files actually exist within the backup.
02-13-2013 07:17 AM
Great idea. The Catalog tapes had the 2 stages backed up, but the tapes were owned by another media server, so even I changed the hostname in the DR file - did not help, but after changing the Media Host Override to the DR media server - the Recovery completed successfully with EMM and NBDB recovered.
Thank you all for the help
02-13-2013 12:23 PM
Ahh, the DR environment did not match the 'original' ...
Still, fixed, which is the main thing.
Martin