Often times when we are trying to restore extremely old data in Backup Exec we encounter all sorts of problems. The causes can vary from a corrupt database or catalogs to media label conflicts within the catalog. Common symptoms are continuously failing catalog jobs and restores failing with inconsistent media errors. The best way I have found to deal with this situation is to revert the BE DB to a blank/clean one temporarily, performing the restore, and then putting everything the way it was before.
This procedure is written from the perspective of Backup Exec 10d, but with some minor adjustments it can easily be performed in other versions.
We start by stopping the BE services. There’s many ways to do this, but the easiest way is from the console (Tools -> Backup Exec Services -> Stop all services). Once the services are stopped we open up “My Computer” and browse over to the location of the “Catalogs” folder (inside of the Backup Exec folder by default, unless you’ve moved it to another location). So if you have the defaults browse over to “C:\Program Files\VERITAS\Backup Exec\NT “on 10d (on 11 and 12 I believe the path is C:\Program Files\Symantec\Backup Exec\). Now rename the “Catalogs” folder to “Catalogs.old”. This makes way for a new, clean catalog.
Next we start up the BEUtility by double clicking on the file within the Backup Exec folder (BEUtility.exe). Click yes to bypass the warning. Click on “All Media Servers”. If you don’t see the name of your backup server on the right hand column, then right click on “All Media Servers”, click on New Media Server, type in the name of the server, and click OK. Now the server should appear on the right. Right click on it and select “Dump Database” from the menu. This creates a backup of your BE DB. Once it completes, click Close.
This next step can save us a lot of pain and sweat in the future. It’s important we create a secondary copy of the DB backup. The reason is that the database maintenance that runs automatically every night will create a new backup of the DB and overwrite the old one. So if the maintenance happens to run before we put everything back our DB backup will be overwritten taking all the backup jobs and schedules with it. It’s possible to recover from this by restoring the DB from backups if we’ve been diligently backing up the Backup Exec folder during our normal backups. But this extra step will save us from that hassle. So we browse over to the “Data” folder within the Backup Exec folder. This folder contains all the DB files as well histories and logs. The backup of the DB that we just created is called “BEDB.bak”. We create a duplicate of it by dragging it and holding the Ctrl key onto its folder. You should see a new file called “Copy of BEDB.bak”.
We are now ready for the clean DB. We go back to BEUtility. Right click on the server name again, and select “Recover Database”. Click on the third recovery method: “Drop existing database and reload from base”. Click OK and then click Yes. It will remove your existing DB and replace it with a new one and then restart the services.
Go ahead and open up Backup Exec. Since this is a new DB you will need to go through the Add New Logon Account Wizard. We are now ready for the restoring part. You might need to reconfigure your devices depending on the media that you intend to restore from (such as adding in backup-to-disk folders). Once you can see the media, inventory all of them, and then catalog them one by one. If you are still having problems running the catalog jobs you can try going to Tools->Options->Catalogs and uncheck “Use storage media-based catalogs”. Then rerun your catalog jobs. If that still does not work, see Appendix 1.
So if all went well, you should have been able to restore your data by this point. Now we need to put everything back so that our backup server can return to normality. Go ahead and stop the services again. Then delete the Catalogs folder (or rename it if you want to save the catalogs from your restore media) and rename the “Catalogs.old” folder to “Catalogs”. Go back into the BEUtility. Right click on your backup server and select “Recover Database” once again. This time select “Drop existing database and reload from backup”. Once it finishes we’re done! We can go back into Backup Exec and verify that our backup jobs are back.
Appendix 1 – Stubborn Catalog Jobs
If your catalog jobs are still failing after doing all the above, try as a last resort. Basically what we are doing is running the catalog job at a low level, byte by byte, sector by sector, rather than just reading the catalog on the media. This is only recommended if you are comfortable making changes to the Windows registry.
I would of course recommend making a backup of the registry first. Next we open up the registry (Start->Run and type in regedit). Open up this entry: “HKEY_LOCAL_MACHINE\SOFTWARE\VERITAS\Backup Exec\Engine\NTFS” and change the value of the FsUseAsyncIo key to ‘0’. Next go to “HKLM\SOFTWARE\VERITAS\Backup Exec\Engine\Tape Format” and change both of the following keys also to ‘0’: “Use Fast Append” and “Use Fast File Restore”. Once these registry changes are done we need to restart the BE services so they go into effect.
Once you go back into Backup Exec, make sure the “Use Storage Media-Based Catalogs” option is unchecked. If the media you are trying to catalog is disk based, there is one last step. Under the Devices tab, right click on the Backup-to-Disk folder which holds the restore media. Select Properties and then the Configuration tab. Under Device settings uncheck “Auto detect settings” and then make sure “Buffered reads” and “Buffered writes” are also unchecked. Under Concurrent operations allow only 1 concurrent operation.
Finally rerun your catalog jobs, one by one. The above steps will ensure that a low level catalog is done. Barring any physical damage or bad sectors on the media, this should work. Once you’re done undo everything you’ve done. Set the 3 registry keys back to ‘1’ and check off “Auto detect settings” in within the B2D folder properties if necessary.