cancel
Showing results for 
Search instead for 
Did you mean: 

Import physical tape with error 172

Wolaf25
Level 4

Hi team


I'm trying to import 3 physical tapes (LTO3), master is NBU 7.5 over linux redhat, using IBM ts3100.
 

The tapes was created using this configuration


Backup was made with Netbackup 7.0, using VTL DL4100.

EDL4100 duplicate the tapes into Adic i500 (drive IBM Ultrium)

Now, I need import these tapes.

When i run the 1st fase for import this error appears

04/14/2016 12:28:03 - Info bptm (pid=17748) Waiting for mount of media id CA1065 (copy 1) on server lacarglxap006.
04/14/2016 12:28:03 - started process bptm (pid=17748)
04/14/2016 12:28:03 - mounting CA1065
04/14/2016 12:28:03 - Info bptm (pid=17748) INF - Waiting for mount of media id CA1065 on server lacarglxap006 for reading.
04/14/2016 12:28:03 - granted resource  CA1065
04/14/2016 12:28:03 - granted resource  IBM.ULT3580-HH5.003
04/14/2016 12:29:32 - mounted CA1065; mount time: 0:01:29
04/14/2016 12:29:32 - Error bptm (pid=17748) block read is not a NetBackup or Backup Exec media header, len = 65536, media id CA1065, drive index 41, data is unknown
04/14/2016 12:29:32 - Info bptm (pid=17748) EXITING with status 172 <----------
04/14/2016 12:29:32 - Error bpimport (pid=11392) Status = cannot read media header, may not be NetBackup media or is corrupted.
04/14/2016 12:29:32 - end Import; elapsed time 0:08:37
cannot read media header, may not be NetBackup media or is corrupted  (172)

but if use the command, previosuly tape monted

/opt/openv/volmgr/bin/scsi_command -map -f /dev/nst0

I can see some dates and some imagenes for the tape

00000000: file 1: record 1: size 65536
01555929: file 1: eof after 1555929 records: 101969362944 bytes
01555930: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 2: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
01555931: file 2: record 2: size 65536
 

As I can understand, the tape is good .

What more I need to do for import these tapes?

Thanks

 

 

 

 

30 REPLIES 30

Michael_G_Ander
Level 6
Certified

Stumbled on to this tech note https://www.veritas.com/support/en_US/article.000024136 about 172 caused by wrong density, while searching for something else on the support site

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

mph999
Level 6
Employee Accredited

I don't  know for sure, but file one record one, if a VTL header would be followed by a tape mark.

Good find on the TN, however again I think not the case here.  QCART is a special case and writes in fixed, I think 512 byte blocks - here the tape is not written in fixed length blocks as the backup header we see is 1k.

I'll ask a colleague to take a look at this, at the moment, :

1.  Try tar - can we confirm what the data is at the beginning of the tape

2.  Is the original tape available can we check that.

As mentioned, my concern is that we are blaming the VTL unfairly, the issue 'might' have happened on the original tape (hence step 2)  and because it was discovered after the VTL duplicate teh tape 'outside NBU' - which is a little unusually and technically 'not supported' it is easy to jump to the wrong conclusion.

Rule one for troubleshooting - keep an open mind, it might not be what you think and 'anything' is possible until proven otherwise.

Even though duplicated outsoide NBU, the tape should still import into a differntent NBU environment providing it is a perfect copy AND the original tape has the backup header.

Also, another thought, would a VTL header not be on an area of the 'tape' that isn't read by NBU ... and as this is a copy to physical, there should be no VTL header at all.

markh794
Level 3
Employee Accredited Certified

The physical tape format is wrong... I'm assuming the original VTL media is no longer available ?

If the original VTL media is still available, try copying the VTL tape to physical tape using 'tcopy' - lots of copies out there in the wild, it's a native BSD utility. (or use NetBackup to duplicate the tape)

A data recovery company may be able to help if the source VTL media isn't available.

Or you could try to 'copy' the tape again, but this time make sure the first block is 1k in size.. otherwise NetBackup will reject the tape as a non-NetBackup format.

 

Can you 'dd' the first block off tape into a file ?

e.g. 'dd if=/dev/nstX of=/tmp/header bs=64k count=1'

Then perform a 'strings /tmp/header' and confirm the media label looks sane.

 

If the media header is there, grab source of tcopy.c, modify so the first block only writes out 1k and recompile. tcopy the tape to another tape and try importing the new copy.

Until that media header is 1k, NetBackup logic to detect valid media format will reject this tape.

It looks like a bug with the VTL native copy utility and I would suspect all such copied media will be corrupt.

mph999
Level 6
Employee Accredited

Thanks Mark - much appreciated.

Wolaf25
Level 4

Michael, yes, was written by EDl4100, using native exportations. On the tape, mouse right buttom and export. The density is correct.

Markh794, virtual tape media was erased long time ago, also EDL4100 was destroyed.
I will try to do what you recommend, here take time for complete the permission for execute the procedure.

For all, I execute SCSI_COMMAND -MAP and this was the result for the 3 tapes with issue

cat /tmp/CA1065
00000000: file 1: record 1: size 65536
01555929: file 1: eof after 1555929 records: 101969362944 bytes
01555930: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 2: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
01555931: file 2: record 2: size 65536
02840465: file 2: eof after 1284535 records: 84183221248 bytes
02840466: file 3: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 3: file 3: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
02840467: file 3: record 2: size 65536
04347023: file 3: eof after 1506557 records: 98733655040 bytes
04347024: file 4: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 4: file 4: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
04347025: file 4: record 2: size 65536
05771717: file 4: eof after 1424693 records: 93368615936 bytes
05771718: file 5: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 5: file 5: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
05771719: file 5: record 2: size 65536
07509136: file 5: eof after 1737418 records: 113863361536 bytes
07509137: file 6: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 6: file 6: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
07509138: file 6: record 2: size 65536
07725381: file 6: eof after 216244 records: 14171702272 bytes
eot


contenido tape CA1066

 cat /tmp/CA1066
00000000: file 1: record 1: size 65536
01273645: file 1: eof after 1273645 records: 83469598720 bytes
01273646: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 8: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1066
01273647: file 2: record 2: size 65536
03052345: file 2: eof after 1778699 records: 116568753152 bytes
03052346: file 3: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 9: file 3: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1066
03052347: file 3: record 2: size 65536
04541481: file 3: eof after 1489135 records: 97591886848 bytes
04541482: file 4: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 10: file 4: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1066
04541483: file 4: record 2: size 65536
05939257: file 4: eof after 1397775 records: 91604517888 bytes
05939258: file 5: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 11: file 5: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1066
05939259: file 5: record 2: size 65536
07258097: file 5: eof after 1318839 records: 86431368192 bytes
07258098: file 6: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 12: file 6: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1066
07258099: file 6: record 2: size 65536
07737380: file 6: eof after 479282 records: 31410160640 bytes
eot

Contenido tape A1068 cat /tmp/CA1068
00000000: file 1: record 1: size 65536
00951648: file 1: eof after 951648 records: 62367203328 bytes
00951649: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 14: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media E00207
00951650: file 2: record 2: size 65536
00984618: file 2: record 32970: size 55296
00984619: file 2: eof after 32970 records: 2160647168 bytes
00984620: file 3: record 1: size 1024: NBU TIR header
          backup_id lacargbaas099_1396049286: frag -1: file 3: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media E00207
          expected size 129233109
00984621: file 3: record 2: size 65536
00986592: file 3: record 1973: size 62464
00986593: file 3: eof after 1973 records: 129234944 bytes
00986594: file 4: record 1: size 1024: NBU EMPTY header (file 4)
00986595: file 4: eof after 1 records: 1024 bytes
eot

Something is not good here ...

I saw 00951649: file 2: record 1: size 1024: NBU BACKUP header
 

Is it NBU Header or not?

Is I use tar I saw

/opt/openv/netbackup/bin/tar tvf /dev/nst0
Blocksize = 128 records
Hmm, this doesn't look like a tar archive.
Skipping to next file header...

Is a inconsistency?

Maybe my skills are not enauf for understand this issue ...

Any Comments?

Thanks to all !!!!

 

 

 

 

 

Michael_G_Ander
Level 6
Certified

Think you could read the 2nd file and forward on these tapes by setting the block size (b) on tar to 128 (65536/512)

I have an idea that the Netbackup tape header and the first Netbackup Image header is inside the first 64KB record.

I would try use dd with ibs=64K and different obs values starting with 1K on the first 64K record, too see if I could find something looking like a Netbackup header

Thinking something like dd if=/dev/rmt/<tape dev> ibs=64K count=1 obs=1K | od -cx

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

mph999
Level 6
Employee Accredited

I saw 00951649: file 2: record 1: size 1024: NBU BACKUP header

This is correct, there is a backup header at the start of each image on the tape (see my diagram above) - therefore a tape will have 1 or more backup headers

There should be a media header at the beginning, this is 'missing' - a tape will only ever have 1 media header.

eg.

MH BH DATA DATA DATA BH DATA BH DATA DATA DATA DATA DATA DATA BH DATA EH

(More DATAs mean a bigger backup, EH = Empty header at the end)

The EH is used to locate the end of the last backup, so we can append more backups later.  A tape that spans to another (backup was too big for the tape) will not have an EH, but we don't need it as we'll never append to a full tape.

Without the MH (media header) NBU cannot mount the tape.

Try Marks suggestion (he wrote MHVTL and is one of, if not the, most knowledgeable people in the company regarding VTLs/ tapes (and other things as well, he's awesome)..

I have to admit, that if Mark is correct, I never thought the VTL copy could screw things up that much.

markh794
Level 3
Employee Accredited Certified

In front of a computer instead of a mobile device..

 

Here is what the process should look like.

Mount tape with media ID J00008 of hcart density

# tpreq -m J00008 -d hcart -p NetBackup -f /tmp/fred

Read in the first block of data (Media Header) - Attempt to read in 64k but we all know it should only read in 1k

# dd if=/tmp/fred bs=64k count=1 of=/tmp/media_header
0+1 records in
0+1 records out
1024 bytes (1.0 kB) copied, 0.00071469 s, 1.4 MB/s

Now use hexdump to view the contents

# hexdump -C /tmp/media_header 
00000000  56 4f 4c 31 4a 30 30 30  30 38 00 00 00 00 00 00  |VOL1J00008......|
00000010  00 00 00 00 00 00 00 01  00 00 00 06 57 1c 73 a1  |............W.s.|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 01  |................|
00000040  00 00 00 00 00 00 00 00  00 00 04 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000070  54 68 49 73 20 49 73 20  41 20 42 50 20 74 41 70  |ThIs Is A BP tAp|
00000080  45 20 68 45 61 44 65 72  00 00 00 00 00 00 00 00  |E hEaDer........|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400

And clean up afterwards - unmount the tape

# tpunmount /tmp/fred

markh794
Level 3
Employee Accredited Certified

I have managed to 'reproduce' the problem by some use of 'dd' and 'tcopy' (if you have problems finding a copy - download the latest copy of 'mhVTL' - www.mhvtl.com. There is a copy of the tcopy utility included with the source).

Mount the source media - I'm choosing /tmp/corrupt (but any filename that does *not* exist will do)

# tpreq -m J00001 -d hcart -p NetBackup -f /tmp/corrupt

# scsi_command -d /tmp/corrupt -map
00000000: file 1: record 1: size 65536              <==== 64k not a valid NetBackup media header
00000001: file 1: eof after 1 records: 65536 bytes
00000002: file 2: record 1: size 1024: NBU BACKUP header
          backup_id mhnbu75_1461482401: frag 1: file 1: copy 1
          expiration 1462087201: retention 0: block_size 524288
          flags 0x0: mpx_headers 0: resume_count 0: media J00008
00000003: file 2: record 2: size 524288
00000123: file 2: record 122: size 65536
00000124: file 2: eof after 122 records: 62981120 bytes
00000125: file 3: record 1: size 1024: NBU EMPTY header (file 2)
00000126: file 3: eof after 1 records: 1024 bytes
00000127: file 4: eof after 0 records: 0 bytes
eot

Move the new tape into a pool (so it won't be chosen by NetBackup for fresh backups) - I'm using the 'None' pool in this example:

# vmchange -m J00002 -p 0

Mount media J00002 - I'm choosing /tmp/fix (but any filename that does *not* exist will do)

# tpreq -d hcart -m J00002 -p None -f /tmp/fix

Just to be sure - rewind both tapes:

# mt -f /tmp/corrupt rewind
# mt -f /tmp/fix rewind

Read in the first block of data - we already know it is 64k

# dd if=/tmp/corrupt of=/tmp/blk_header bs=64k count=1
1+0 records in
1+0 records out
65536 bytes (66 kB) copied, 0.00105776 s, 62.0 MB/s

Confirm the first 1k looks OK (if not, it's time to have a chat with a data recovery house - sorry).

This exampe I had 'corrupted' the 1k block with additional 63k of NULLs - but it's the data between 00000000 to 00000090 that we're concerned with.

# hexdump -C /tmp/blk_header 
00000000  56 4f 4c 31 4a 30 30 30  30 38 00 00 00 00 00 00  |VOL1J00008......|
00000010  00 00 00 00 00 00 00 01  00 00 00 06 57 1c 73 a1  |............W.s.|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 01  |................|
00000040  00 00 00 00 00 00 00 00  00 00 04 00 00 00 00 00  |................|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000070  54 68 49 73 20 49 73 20  41 20 42 50 20 74 41 70  |ThIs Is A BP tAp|
00000080  45 20 68 45 61 44 65 72  00 00 00 00 00 00 00 00  |E hEaDer........|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00010000

Now read the first 1k of data from /tmp/blk_header to the tape mounted as /tmp/fix

# dd if=/tmp/blk_header of=/tmp/fix bs=1k count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 0.00668387 s, 153 kB/s

The above 'dd' command will write the 1k of data plus the filemark and will be positioned at the EOT (End of Tape) side of the filemark.

Lets position the source media in the same spot (Note: I'm using the 'mt' command from the optional mt-st RPM package):

# mt -f /tmp/corrupt rewind
# mt -f /tmp/corrupt fsf
# mt -f /tmp/corrupt tell
At block 2.

Now use the 'tcopy' utility to copy from source (/tmp/corrupt) to destination (/tmp/fix)

# tcopy /tmp/corrupt /tmp/fix
file 0: block size 1024: record 0
file 0: block size 524288: records 1 to 121
file 0: block size 65536: record 121
file 0: eof after 122 records: 62981120 bytes
file 1: block size 1024: 1 records
file 1: eof after 1 records: 1024 bytes
eot
total length: 62982144 bytes

Time for  a cup of coffee while the tape contents is copied - it is typical to take about 2hrs to read a tape from end-to-end ;)

Now lets check the final result:

# mt -f /tmp/fix rewind
# scsi_command -d /tmp/fix -map
00000000: file 1: record 1: size 1024: NBU MEDIA header (J00008)    <=== With luck, this is now reported as a NBU Media header
00000001: file 1: eof after 1 records: 1024 bytes
00000002: file 2: record 1: size 1024: NBU BACKUP header
          backup_id mhnbu75_1461482401: frag 1: file 1: copy 1
          expiration 1462087201: retention 0: block_size 524288
          flags 0x0: mpx_headers 0: resume_count 0: media J00008
00000003: file 2: record 2: size 524288
00000123: file 2: record 122: size 65536
00000124: file 2: eof after 122 records: 62981120 bytes
00000125: file 3: record 1: size 1024: NBU EMPTY header (file 2)
00000126: file 3: eof after 1 records: 1024 bytes
00000127: file 4: eof after 0 records: 0 bytes
eot

Unmount source and destination tapes:

# tpunmount -f /tmp/corrupt
# tpunmount -f /tmp/fix

And repeat the process for your other tapes.. This email thread shows you have at least three tapes for backup image - lacargbaas099_1396049286

Please post an update so we know how you get on.

markh794
Level 3
Employee Accredited Certified

One follow up comment - Reading thru the thread again, I noticed you needed change control to have some (mt) packages installed.

My recommended minimum set of RPM packages to troubleshoot SCSI devices (including disk, tape and libraries) on Linux are:

- mt-st     (for tape drives)
- mtx       (for tape libraries)
- sg3_utils (for all SCSI devices)
- lsscsi    (for all SCSI devices)

Wolaf25
Level 4

Wow ! a lot new information for me.

I will try to do what are you suggested. I havn't free time in this moments.

I'll keep informed to all.