07-16-2012 11:50 AM
Previously with netbackup 6.5 we had all offline backups, this was discontinued with version 7.
with netbackup version 6.5 we could run the following command to recover the catalog
/usr/openv/netbackup/bin/admincmd/bprecover -r -dpath <path>
now with netbackup version 7.5.0.3 we are trying to import the old media, this should be a multistage process however all the tapes appear to be failing with error 172. Are we performing the steps wrong?? This is being tested on a clean server with no image database with a fresh install of 7.5.0.3 and not an upgrade.
admincmd/bpimport -create_db_info -id 1174L3 -server usadmnb01p
Import phase 1 started Mon Jul 16 14:30:48 2012
INF - Create DB information for media id 1174L3.
INF - Initiation of bptm process to phase 1 import media id 1174L3 was successful.
INF - Status = cannot read media header, may not be NetBackup media or is corrupted.
INF - Consult the Activity Monitor for more information about this job.
Thanks
07-16-2012 12:13 PM
Here you are talking about catalog recovery.
/usr/openv/netbackup/bin/admincmd/bprecover -r -dpath <path>
(the command line equiv. of running the catalog recovery wizard)
You need to be aware that this has nothing to do with importing media.
Here is the potential issue:
INF - Status = cannot read media header, may not be NetBackup media or is corrupted.
07-16-2012 12:23 PM
m s2 d1
Initiating MOVE_MEDIUM from address 4097 to 256
MOVE_MEDIUM complete
# mt -f /dev/rmt/1cbn rew
# od -c /dev/rmt/1cbn
0000000 V E R S I O N 1 U N C O M P
0000020 R E S S E D \n u s a d m n b 0 1
0000040 p \n 1 1 7 4 L 3 \n 0 5 / 1 3 / 1
0000060 0 2 2 : 3 8 : 1 8 \n 3 2 7 6 8
0000100 \n 4 6 \n I M A G E 1 = U S A
0000120 D M N B 0 1 P : / u s r / o p e
0000140 n v / n e t b a c k u p / d b \n
0000160 I M A G E 2 = U S A D M N B
0000200 0 1 P : / u s r / o p e n v / v
0000220 o l m g r / d a t a b a s e \n I
0000240 M A G E 3 = U S A D M N B 0
0000260 1 P : / u s r / o p e n v / v a
0000300 r / a u t h \n I M A G E 4 =
0000320 U S A D M N B 0 1 P : / u s r /
0000340 o p e n v / v a r / g l o b a l
0000360 \n I M A G E 5 = U S A D M N
0000400 B 0 1 P : / u s r / o p e n v /
0000420 v a r / v x s s \n I M A G E 6
0000440 = U S A D M N B 0 1 P : / u s
0000460 r / o p e n v / v a r / l i c e
0000500 n s e . t x t \n I M A G E 7 =
0000520 U S A D M N B 0 1 P : / u s r
0000540 / o p e n v / v a r / n b p r o
0000560 x y _ n b s l - M a n a g e M e
0000600 d i a - 2 . i o r . m g r \n I M
0000620 A G E 8 = U S A D M N B 0 1
0000640 P : / u s r / o p e n v / v a r
0000660 / n b p r o x y _ n b s l - M a
0000700 n a g e M e d i a - 1 . i o r .
0000720 m g r \n I M A G E 9 = U S A
0000740 D M N B 0 1 P : / u s r / o p e
0000760 n v / v a r / n b p r o x y _ n
0001000 b s l - M a n a g e M e d i a -
0001020 5 . i o r . m g r \n I M A G E 1
0001040 0 = U S A D M N B 0 1 P : /
0001060 u s r / o p e n v / v a r / n b
0001100 d e l e t e \n I M A G E 1 1 =
0001120 U S A D M N B 0 1 P : / u s r
0001140 / o p e n v / v a r / n b p r o
0001160 x y _ n b s l - D e v i c e M a
0001200 n a g e m e n t - 3 . i o r . m
0001220 g r \n I M A G E 1 2 = U S A
0001240 D M N B 0 1 P : / u s r / o p e
0001260 n v / v a r / a u t h o r i z e
0001300 . t x t \n I M A G E 1 3 = U
0001320 S A D M N B 0 1 P : / u s r / o
0001340 p e n v / v a r / n b p r o x y
0001360 _ n b s l - M a n a g e M e d i
0001400 a - 3 . i o r . m g r \n I M A G
0001420 E 1 4 = U S A D M N B 0 1 P
0001440 : / u s r / o p e n v / v a r /
0001460 n b p r o x y _ n b s l - M a n
0001500 a g e M e d i a - 8 . i o r . m
0001520 g r \n I M A G E 1 5 = U S A
0001540 D M N B 0 1 P : / u s r / o p e
0001560 n v / v a r / n b p r o x y _ n
0001600 b s l - M a n a g e M e d i a -
0001620 4 . i o r . m g r \n I M A G E 1
0001640 6 = U S A D M N B 0 1 P : /
0001660 u s r / o p e n v / v a r / n b
0001700 p r o x y _ n b s l - M a n a g
0001720 e M e d i a - 7 . i o r . m g r
0001740 \n I M A G E 1 7 = U S A D M
0001760 N B 0 1 P : / u s r / o p e n v
0002000 / v a r / n b p r o x y _ n b s
0002020 l - M a n a g e M e d i a - 9 .
0002040 i o r . m g r \n I M A G E 1 8
0002060 = U S A D M N B 0 1 P : / u s
0002100 r / o p e n v / v a r / n b p r
0002120 o x y _ n b s l - M a n a g e M
0002140 e d i a - 6 . i o r . m g r \n I
0002160 M A G E 1 9 = U S A D M N B
0002200 0 1 P : / u s r / o p e n v / v
0002220 a r / n b p r o x y _ n b s l -
0002240 M a n a g e M e d i a - 1 1 . i
0002260 o r . m g r \n I M A G E 2 0 =
0002300 U S A D M N B 0 1 P : / u s r
0002320 / o p e n v / v a r / n b p r o
0002340 x y _ n b s l - M a n a g e M e
0002360 d i a - 1 2 . i o r . m g r \n I
0002400 M A G E 2 1 = U S A D M N B
0002420 0 1 P : / u s r / o p e n v / v
0002440 a r / n b p r o x y _ n b s l -
0002460 M a n a g e M e d i a - 1 0 . i
0002500 o r . m g r \n I M A G E 2 2 =
0002520 U S A D M N B 0 1 P : / u s r
0002540 / o p e n v / v a r / n b p r o
0002560 x y _ n b s l - M a n a g e M e
0002600 d i a - 1 4 . i o r . m g r \n I
0002620 M A G E 2 3 = U S A D M N B
0002640 0 1 P : / u s r / o p e n v / v
0002660 a r / n b p r o x y _ n b s l -
0002700 M a n a g e M e d i a - 1 3 . i
0002720 o r . m g r \n I M A G E 2 4 =
0002740 U S A D M N B 0 1 P : / u s r
0002760 / o p e n v / v a r / n b p r o
0003000 x y _ n b s l - M a n a g e M e
0003020 d i a - 1 7 . i o r . m g r \n I
0003040 M A G E 2 5 = U S A D M N B
0003060 0 1 P : / u s r / o p e n v / v
0003100 a r / n b p r o x y _ n b s l -
0003120 M a n a g e M e d i a - 1 5 . i
0003140 o r . m g r \n I M A G E 2 6 =
0003160 U S A D M N B 0 1 P : / u s r
0003200 / o p e n v / v a r / n b p r o
0003220 x y _ n b s l - D e v i c e M a
0003240 n a g e m e n t - 5 . i o r . m
0003260 g r \n I M A G E 2 7 = U S A
0003300 D M N B 0 1 P : / u s r / o p e
0003320 n v / v a r / n b p r o x y _ n
0003340 b s l - M a n a g e M e d i a -
0003360 1 6 . i o r . m g r \n I M A G E
0003400 2 8 = U S A D M N B 0 1 P :
0003420 / u s r / o p e n v / v a r / n
0003440 b p r o x y _ n b s l - D e v i
0003460 c e M a n a g e m e n t - 9 . i
0003500 o r . m g r \n I M A G E 2 9 =
0003520 U S A D M N B 0 1 P : / u s r
0003540 / o p e n v / v a r / n b p r o
0003560 x y _ n b s l - M a n a g e M e
0003600 d i a - 1 8 . i o r . m g r \n I
0003620 M A G E 3 0 = U S A D M N B
0003640 0 1 P : / u s r / o p e n v / v
0003660 a r / n b p r o x y _ n b s l -
0003700 D e v i c e M a n a g e m e n t
0003720 - 8 . i o r . m g r \n I M A G E
0003740 3 1 = U S A D M N B 0 1 P :
0003760 / u s r / o p e n v / v a r / n
0004000 b p r o x y _ n b s l - M a n a
0004020 g e M e d i a - 1 9 . i o r . m
0004040 g r \n I M A G E 3 2 = U S A
0004060 D M N B 0 1 P : / u s r / o p e
0004100 n v / v a r / n b p r o x y _ n
0004120 b s l - M a n a g e M e d i a -
0004140 2 0 . i o r . m g r \n I M A G E
0004160 3 3 = U S A D M N B 0 1 P :
0004200 / u s r / o p e n v / v a r / n
0004220 b p r o x y _ n b s l - D e v i
0004240 c e M a n a g e m e n t - 1 3 .
0004260 i o r . m g r \n I M A G E 3 4
0004300 = U S A D M N B 0 1 P : / u s
0004320 r / o p e n v / v a r / n b p r
0004340 o x y _ n b s l - M a n a g e M
0004360 e d i a - 2 1 . i o r . m g r \n
0004400 I M A G E 3 5 = U S A D M N
0004420 B 0 1 P : / u s r / o p e n v /
0004440 v a r / n b p r o x y _ n b s l
0004460 - M a n a g e M e d i a - 2 4 .
0004500 i o r . m g r \n I M A G E 3 6
0004520 = U S A D M N B 0 1 P : / u s
0004540 r / o p e n v / v a r / n b p r
0004560 o x y _ n b s l - M a n a g e M
0004600 e d i a - 2 2 . i o r . m g r \n
0004620 I M A G E 3 7 = U S A D M N
0004640 B 0 1 P : / u s r / o p e n v /
0004660 v a r / n b p r o x y _ n b s l
0004700 - M a n a g e M e d i a - 2 3 .
0004720 i o r . m g r \n I M A G E 3 8
0004740 = U S A D M N B 0 1 P : / u s
0004760 r / o p e n v / v a r / n b p r
0005000 o x y _ n b s l - M a n a g e M
0005020 e d i a - 2 5 . i o r . m g r \n
0005040 I M A G E 3 9 = U S A D M N
0005060 B 0 1 P : / u s r / o p e n v /
0005100 v a r / n b p r o x y _ n b s l
0005120 - M a n a g e M e d i a - 2 6 .
0005140 i o r . m g r \n I M A G E 4 0
0005160 = u s a d m n b 0 1 p : / u s
0005200 r / o p e n v / d b / d a t a /
0005220 N B D B . d b \n I M A G E 4 1
0005240 = u s a d m n b 0 1 p : / u s
0005260 r / o p e n v / d b / d a t a /
0005300 E M M _ D A T A . d b \n I M A G
0005320 E 4 2 = u s a d m n b 0 1 p
0005340 : / u s r / o p e n v / d b / d
0005360 a t a / E M M _ I N D E X . d b
0005400 \n I M A G E 4 3 = u s a d m
0005420 n b 0 1 p : / u s r / o p e n v
0005440 / d b / d a t a / v x d b m s .
0005460 c o n f \n I M A G E 4 4 = u
0005500 s a d m n b 0 1 p : / u s r / o
0005520 p e n v / v a r / g l o b a l /
0005540 s e r v e r . c o n f \n I M A G
0005560 E 4 5 = u s a d m n b 0 1 p
0005600 : / u s r / o p e n v / v a r /
0005620 g l o b a l / d a t a b a s e s
0005640 . c o n f \n I M A G E 4 6 =
0005660 u s a d m n b 0 1 p : / u s r /
0005700 o p e n v / d b / d a t a / N B
0005720 D B . l o g \n \0 x 004 f 374 177 375 377 377
0005740 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0005760 220 277 y \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
0006000
07-16-2012 01:09 PM
Blimy ... that is NOT a Netbackup header.
That tape is NOT recoverable via NBU as there is no header.
I suspect the tape has been rewound via an 'external' event while it was backing up.
1. If you have multiple devices accessing the tape drives (eg. NDMP filers) make sure that the scsi reservation type set in NBU (in host properties) is set the same on the filers.
2. If you have no NDMP filers sharing the drives, then the most likely cause of this issue is ANYTHING between the OS and the drives.
Tape drivers / Drive firmware
HBA hardware / Driver / Firmware
Switch / Switch Firmware
Hardware fault on any of the above etc ...
Unfortunately, if this is a scsi issue, the one thing it cannot be is NBU.
From TN http://www.symantec.com/docs/TECH169477
07-17-2012 06:17 AM
How is it 6.5 has no problem reading in a offline catalog, yet 7.5 has issues. Or am I doing something wrong to properly read in an offline catalog.
# pkgrm SYMCnbclt SYMCnbjava SYMCnbjre SYMCnetbp
# rm -rf /gtnbuild/openv
# mkdir /gtnbuild/openv
# /export/zone1/root/65_solaris_x86/install (answer Y all questions and reinstall 6.5)
# cat /usr/openv/netbackup/bin/version
NetBackup-Solaris_x86_10_64 6.5
reconfigure robot using jnbSA
run inventory
# ls -al /usr/openv/netbackup/db/images/ | wc -l
4
# /usr/openv/netbackup/bin/admincmd/bprecover -r -m 1174L3 -d hcart3
Recover USADMNB01P:/usr/openv/netbackup/db y/n (n)? y
Recovering USADMNB01P:/usr/openv/netbackup/db
(answer y like 20 times)
# ls -al /usr/openv/netbackup/db/images/ | wc -l
122
# pkgrm SYMCnetbp
Reinstall fresh clean 7.5.0.3 not upgrade
# ./install
# ./NB_update.install
# cat /usr/openv/netbackup/bin/version
NetBackup-Solaris_x86_10_64 7.5.0.3
reconfigure robot using jnbSA
run inventory
# ls -la /usr/openv/netbackup/db/images | wc -l
4
# /usr/openv/netbackup/bin/admincmd/bprecover -r -m 1174L3 -d hcart3
bprecover: unrecognized option -m
bprecover: -wizard [-copy <number>]
-r -nbdb [-priority <number>] [-copy <number>] [-L <output_file>]
# /usr/openv/netbackup/bin/admincmd/bprecover -wizard
Welcome to the NetBackup Catalog Recovery Wizard!
Please make sure the devices and media that contain catalog disaster recovery
data are available
Are you ready to continue?(Y/N)
y
Please specify the full pathname to the catalog disaster recovery file:
DOESNT EXIST
07-17-2012 02:36 PM
NBU 7.x cannot read an offline catalog.
A catalog restore can ONLY be performed back to exactly the same version of NBU that created it. This is fully documented.
So a 6.5 catalog backup can only be recovered to a 6.5 server.
Now, I see, that tape is probably a cold catalog tape - sorry I dodn't spot this before, I've never seen the od output on one. Normally, when a tape is missing the normal NBU media header it is because it is overwritten. In this case, I see my mistake - it is a cold catalog tape.
So, no, NBU will not read it, and even if it did, is is useless, as NBU 7.x will only recover a hot catalog AND, as this is a cold catalog it cannot have been made at 7.5, and so cannot be restored to a 7.5 system.
Does this clear things up for you ?
Martin
07-17-2012 02:59 PM
From here :
http://www.symantec.com/docs/TECH58358
"Cold catalog backups are not supported starting at NetBackup 7.0"
Martin
07-17-2012 03:08 PM
yes, I can understand removing the ability to perform a cold catalog backup, I DONT understand the removal of the ability to RECOVER a cold catalog backup
now I have to have 1 server running a 6.5 to recover cold catalogs
6.5 is not supported and is EOL
so you can see my chicken and egg dilema
IMHO this is a fundamental flaw of design by Symantec
07-17-2012 03:25 PM
It's not a design flaw, it was documented prior to the release of NBU 7, and it in the NBU 7 documentation.
However, let us not fall out over this, I like to stay friends - I didn't make the rules, and can do nothing about it.
I think there is a mis-understanding.
A cold catalog tape must have been made at a version lower than 7, and from your post I see this is 6.5
At 7, you make a new catalog backup, as you aware.
What good therefore, is the old catalog backup (cold) made at 6.5 .
It can never be used to restore a 7.x server and is out-of-date the minute you made a new catlog backup at 7.5
The only use it had was in case you need to recover the 6.5 server if the upgrade failed.
Once at 7.5, it is out of data and you have replaced it when the lated copy made.
There is nothing on that tape that is any use to you.
Therefore I understand your initial concern that the tape isn't reaable, but even if it was, it is of no use as it cannot be used.
So, yes, if you wanted to recover the 6,5 catalog, you would need a 6.5 server, that had always been the case, and would be true if you had a hot 6.5 catalog backup.
Martin
Martin
07-17-2012 03:59 PM
Can we take a step back and determine exactly what it is you are trying to do? It sounds like you are wanting to import your old 6.5 catalog information into a new 7.5 test system? For what pupose? Are you going to do your "production" install to a new clean environment then try and import all your old data also? What about upgrading the 6.5 to 7.5 in place so you don't have to do this? If you need to use new servers for your master and media servers I would suggest you get with professional services to get their help. To the best of my knowledge the only supported way to "merge and/or migrate" catalogs is with their help.
07-17-2012 04:33 PM
js88699 - no need, I think amado is just importing the old media onto a new server, without the catalog recovery.
This is fine, a little slow but fine. Actually it is quite neat, as you end up with the old media imported, but as this rebuilds tha catalog, it comes back nice and clean.
The only issue I see is the mis-understanding about the cold catalog tapes. These are not needed as they have no use at 7 as I have explained.
Martin
07-17-2012 07:39 PM
Instead of stepping back lets step forward...
it is now the year 2015 and my compliance department/auditors have requested trading data from January of 2012
now I am scrambling to find a server and a tape unit as all I have ever done was cold offline tape catalogs
where is the backward compatibility????
This is a design flaw, is it not? Why should I scramble for a new server/new tape unit/massive downtime to be able to restore a 6.5 catalog????
thanks
07-18-2012 01:05 AM
No, as I said it is not a design flaw.
A catalog tape contains nothing of interest to auditors, but, it is important to have it in case of recovery - they will be interested in that, but not what is on it.
Lets take another step back - is this correct ....
1. You have a 6.5 system.
2. You have built a new 7.5 system
3. You have taken the 6.5 tapes and are importig them into 7.5 system
1) If this is not correct, please explain what the 'situation' is.
If this is correct, then importing the tapes into 7.5 has recreated the catalog information. for these tapes.
Therefore, all the catalog information that was on the tapes, is now on the 7.5 system, and, running a catalog backup at 7.5 will create a new catalog backup that contains info on the tapes that have been imported, which is a new version of the cold catalog backup.
As mentioned, the cold tape can only be recovered to a 6.5 system, so what good is reading it anyway on a 7.x system, it can't be used for recovery.
I don't understand why you need to read this tape on a 7 system - the 7 system contains it's on version of the same information.
Perhaps I am mis-understanding, the exact details of this ...
A) Where exactly are your tapes ?
B) Why do you need to perfom a catalog recovery - the tapes should be kept in your system, and thus in the catalog.
C) Lets just pretend that you could read this catalog tape on 7.x - what would you do with th info.
I am just trying to understand what you are trying to do.
Tapes once used by NBU should always stay in the catalog. The only time a catalog recovery should be required is in the event of a system failure/ reinstall (for whatever reason).
There should be no case where to get at data on tapes, the catalog should need to be recovered.
If the 6.5 system is still running, why not run a hot catalog backup. This could be imported in to 7.x (imported, NOT recovered). It would be readable, but why you would want to do this I don't know.
Martin
07-18-2012 01:06 AM
I've asked Marainne to look at this. Maybe I misunderstand something.
Martin
07-18-2012 02:50 AM
Due to disk space limitations we would expire data after 1 year, previously when it came around for audit time, we would recall the 2 or 3 tapes from that week that had the catalog, then recall the 1 or 2 tapes that had the data. grand total 5 tapes recalled at most.
now I would have to recall 30-40 tapes and scan all of them to figure out which tape has the data that the auditors are requesting.
Am I missing something here?
07-18-2012 02:58 AM
I am totally with everyone else one this.
There are two ways to recover:
1. Catalog recovery.
Only the latest catalog backup is relevant. If you want to restore 2012 data in 2015, your retention levels should be configured to satisfy the necessary retention requirements.
2015 catalog recovery will contain all the data backed up in 2012 if retention levels are correct.
There has NEVER been a need to recover catalog backup from 3 versions back.
2. Import tapes
If for some or other silly reason no catalog backup was done or incorrect retention levels were set on backup schedules, tapes containing actual data can be imported. Any NBU version data can be imported for as long as NBU will write it's data in tar format (provided physical media is still good and you have a tape drive to read old media).
It makes no sense whatsovever to try and import an old catalog tape....
..... Just my 2c .....
07-18-2012 04:10 AM
Due to disk space limitations we would expire data after 1 year, previously when it came around for audit time, we would recall the 2 or 3 tapes from that week that had the catalog, rename some folders, then recall the 1 or 2 tapes that had the data. grand total 5 tapes recalled at most.
now I would have to recall 30-40 or the entire weekof tapes and scan all of them to figure out which tape has the data that the auditors are requesting.
Am I missing something here?
07-18-2012 04:42 AM
Yes. Configure retention levels correctly.
Retention levels should always match recovery requirements.
There are all sorts of ways to cater for large catalogs:
1) Add more disk space. There are TN's with instructions on how to move catalogs to new location
2) Enable catalog compression (not ideal, but helps with space until more can be added)
3) Do catalog archiving. Instructions in Admin Guide.
One more thought:
With Hot Catalog backups (introduced in NBU 6.0), you can import catalog tapes like any regular tape and restore selected image files....
Cold catalogs are absolute bad news now that we know about Hot Catalogs.
07-18-2012 05:20 AM
Yes you are ...
This is not how to run a backup environment, the system is designed incorrectly.
However, I see the use of the catalog tape.
The correct method is :
A/ Design the system correctly + add more dosk
I would not accept waht you have in a production environment, certainly not in one that is audited.
PLEASE NOTE: I appreciate this in not you personally, I am just pointing it out how it is.
However, as I mentioned, you could create a hot catalog tape.
Marianne will be along in a minute as we had a chat off line with the details, I'll let her do that bit.
But the botom line, is that is a shocking setup, I am sorry, and I do not mean it personally.
Why is this so bad ...
What happens if you expire the wrong tape by accident, or someone puts the wrong tapes back in. There is no protection, to NBU they are new tapes.
I saw an issue 2/3 years back where someone was doing something kind of similar - they made a mistake (which they insisted could never happen) and overwrote a significant amount of tapes, becuase, they expired the wrong ones.
Martin
EDIT.. Catalog archiving would solve the problem. Not ideal, but better than what you have.
07-18-2012 06:18 AM
Be that as it may, due to lack of backward compatability for the offline catalogs, I now have to keep a server in a closet with 6.5 and a DLT drive for possibly the next 20 years (or until they allow us to overwrite those tapes) becuase 7.x doesnt understand what that tape is....or read in a full weeks 40 of tapes in the hope I can find the data im looking for.
While yes Cold catalogs are bad news now, when this system and procedures were drawn up years ago everything worked, and when procedures work people dont like to change. while I did not design the setup, I am working with what I had. we have expanded to an array, and upgraded this does nothing for data in the past, only in the future.
removing the ability to perform it I have no problems with
removing the ability to recover that data is the problem I have