total meltdown - unablle to recover nbdb

[dennis_sexton]

What would you do if you cannot recover the nbdb from the catalog backup.
I'm runnning Netbackup 6.0 MP4 on a solaris 9 machine. W

Doing the steps in the recovery section of the troubleshooting manual, I get this on step 11, page 570.

##> /usr/openv/netbackup/bin/admincmd/bprecover -r -nbdb
ERR - Failed to find last NBDB backup image record for client carnival.Stanford.EDU with policy Catalog_backup (227)

or

#> /usr/openv/netbackup/bin/admincmd/bprecover  -r -tpath /dev/rmt/16cbn 
media or file was not created by bpbackupdb
specified media or path does not contain a valid NB database backup header

I need to have backups starting today. we've been down a week trying to recover.

Any suggestions appreciated. Prayers too :)

[dennis_sexton]

Replying to my own question.

Step 1. Make sure you get the right person in Symantec backline.

Remember I got into this mess after the following documented procedure

 ##> bprecover -wizard

Welcome to the NetBackup Catalog Recovery Wizard!

Please make sure the devices and media that contain catalog disaster recovery data are available
Are you ready to continue?(Y/N)
y
Please specify the full pathname to the catalog disaster recovery file:
/var/tmp/catalog_backup/Catalog_backup_1212604620_FULL
master_host_1212604620
All media resources were located
Do you want to recover the entire NetBackup catalog? (Y/N)
y
Catalog recovery is in progress. Please wait...
ERR - Failed to recover NBDB on carnival.stanford.edu (5)
Failed to recover catalog (5)

In fact all of my catalog files were restored. I just didn't have any nbdb to talk to.

From the  Catalog_backup_1212604620_FULL we knew we wanted tapes 1234 and 1235

We had to do the following

Make bar code rules

Inventory the tapes in the library (1234 and 1235 are in the library)

 vmchange -h master_host -p 1 -m 1234

vmchange -h master_host -p 1 -m 1235

vmquery -assignbyid 1234 DLT2 1 1 1213212328
vmquery -assignbyid 1235 DLT2 1 1 1213212328
bptm -makedbentry -m 1234 -poolnum 1 -den 15
bptm -makedbentry -m 1235 -poolnum 1 -den 15
mkdir /opt3/openv/db/staging

using the gui restore /opt/openv/db/staging/* to /opt3/openv/db/staging/

 /etc/init.d/netbackup stop
bpps -a (to confirm nothing running)

cd /usr/openv/db/data

tar cf db.orig.install.tar ./*

cp  db.orig.install.tar /some/safe/place

rm EMM_DATA.db EMM_INDEX.db NBDB.db NBDB.log vxdbms.conf

 cp -p /opt3/openv/db/staging/* .
chmod 400 EMM_DATA.db
chmod 400 EMM_INDEX.db
chmod 444 NBDB.db
chmod 400 databases.conf
chgrp other *
chgrp bin vxdbms.conf

if server.conf included here compare with /usr/openv/var/global/server.conf

if server.conf is different from  /usr/openv/var/global/server.conf put server.conf in its place

   otherwise just move it to /some/safe/place

mv NBDB.log.1 NBDB.log

/etc/init.d/netbackup start

recovery was successful