cancel
Showing results for 
Search instead for 
Did you mean: 

Import physical tape with error 172

Wolaf25
Level 4

Hi team


I'm trying to import 3 physical tapes (LTO3), master is NBU 7.5 over linux redhat, using IBM ts3100.
 

The tapes was created using this configuration


Backup was made with Netbackup 7.0, using VTL DL4100.

EDL4100 duplicate the tapes into Adic i500 (drive IBM Ultrium)

Now, I need import these tapes.

When i run the 1st fase for import this error appears

04/14/2016 12:28:03 - Info bptm (pid=17748) Waiting for mount of media id CA1065 (copy 1) on server lacarglxap006.
04/14/2016 12:28:03 - started process bptm (pid=17748)
04/14/2016 12:28:03 - mounting CA1065
04/14/2016 12:28:03 - Info bptm (pid=17748) INF - Waiting for mount of media id CA1065 on server lacarglxap006 for reading.
04/14/2016 12:28:03 - granted resource  CA1065
04/14/2016 12:28:03 - granted resource  IBM.ULT3580-HH5.003
04/14/2016 12:29:32 - mounted CA1065; mount time: 0:01:29
04/14/2016 12:29:32 - Error bptm (pid=17748) block read is not a NetBackup or Backup Exec media header, len = 65536, media id CA1065, drive index 41, data is unknown
04/14/2016 12:29:32 - Info bptm (pid=17748) EXITING with status 172 <----------
04/14/2016 12:29:32 - Error bpimport (pid=11392) Status = cannot read media header, may not be NetBackup media or is corrupted.
04/14/2016 12:29:32 - end Import; elapsed time 0:08:37
cannot read media header, may not be NetBackup media or is corrupted  (172)

but if use the command, previosuly tape monted

/opt/openv/volmgr/bin/scsi_command -map -f /dev/nst0

I can see some dates and some imagenes for the tape

00000000: file 1: record 1: size 65536
01555929: file 1: eof after 1555929 records: 101969362944 bytes
01555930: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 2: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
01555931: file 2: record 2: size 65536
 

As I can understand, the tape is good .

What more I need to do for import these tapes?

Thanks

 

 

 

 

30 REPLIES 30

Wolaf25
Level 4

Aditional information

 

for change the media_id, because the media rules are different between old NBU 7.0 and new NBU 7.5 I use the following command

 /opt/openv/volmgr/bin/vmadd  -m CA1065 -mt hcart3 -b WCA01065  -rt tld -rn 3 -rh lacarglxap006 -rc1 11 -p 1

Regards

 

Will_Restore
Level 6

Doesn't look good.  See mph666 reply in this older thread:

https://www.veritas.com/community/forums/block-read-not-netbackup-or-backup-exec-media-header-len-65536-media-id-xxxx#comment-10669311

mph999
Level 6
Employee Accredited

The tape is good, the data on the tape is not - you have Data Loss.

From the SCSI command :

00000000: file 1: record 1: size 65536
01555929: file 1: eof after 1555929 records: 101969362944 bytes
01555930: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 2: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
01555931: file 2: record 2: size 65536

Something has written data at the beginning of the tape, file1, record 1 should be NBU Media Header, of size 1024 bytes.

As per my other post - impossible to tell you why this has happened, unless you have the backup logs and they mention "External rewind".

There are two causes I have seen:

(1)

External event on the SAN causes the drives to rewind mid-backup.  This is invisible to the OS (and NBU) and causes the beginning of the tape to be overwritten.  It's causes by a SAN reset, or if the scsi reservation type is set differently on devices that see the same drives (SSO) - eg. NDMP set to SPC2  reservation, and NetBackup media servers set to Persistent reservation.

(2)

Something outside NBU wrote to the tape.

I'm going to guess your SIZE_DATA_BUFFERS file is set to 65536 (64k), and that cause (1) is the most likely.

Although it can't be detected when it happens (It's a SAN event) when we finish the backup we do check the position of the tape, we know how many blocks have been sent, and therefore where the tape should be positioned.  If where the drive tells us it is positioned does not match where we think it should be, you get the External eveny has caused SCSI rewind) event in bptm log.

If the cause is (1) - extenal SAN event, there is a possibilty many tapes are affected.

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

If my interpretation of the opening post is correct, it seems that the duplication is not done by NetBackup, right? 
Duplication done as a VTL function?

If so - this kind of duplication is not supported as the data is not written by NetBackup.

See this extract from an older version of NBU HCL:

11. NetBackup NDMP Direct Copy with VTL or an embedded NetBackup media server (e.g., EMC DL4000) may be used to copy backup images (not media) from the VTL directly to physical tape or a 2nd VTL.

Copying virtual media to physical media or a 2nd virtual media is not supported by Symantec; any support for this must come from the VTL vendor providing the capability. If the VTL copies virtual media to physical media (or a 2nd VTL), NetBackup is not aware of the physical media or 2nd virtual media and may not be able to recover that data directly from it.

cruisen
Level 6
Partner Accredited

Hello Wolaf25,

Tell me did you upgrade the fW from your robbot during this time?

You should downgrade to when the backup was done previously.

This all what I am saying refers only if the conclusion of Marianne is not correct, and you did the duplication using Netbackup only and not by Third party vendors.

Best regards,

Cruisen

 

mph999
Level 6
Employee Accredited

Firmware etc will not help in this case.

scsi_command was the correct command to run to see what was on the tape.  It is very clear, that there in no NBU Media Header, and the first block is of a block size that would equate to data.

 

Marianne makes a good point about how the duplication was done.  If it was done outside NBU then ???

That said, if the copy made was an exact copy of the tape, then it should import onto a system that has 'never seen that tape before' but it would appear here, that if that was meant to happen, it hasn't.

Wolaf25
Level 4

Hi


Marianne and mph999


Duplication was maded with VTL, no Netbackup, but when we had VTL, we import this tape to a virtual tape again(into VTL ) and then using the virtual for restore information.

Is this posible with Data Domian?


How can I do this? Anyone knows?

 

SIZE_DATA_BUFFERS


actually the size is


$ cat SIZE_DATA_BUFFERS
262144

 

because its a diferente master, the old master was unix 6.1 and the new master is redhat, more power.

I try to use mt command  and it wasn't posible because mt is not installed into the netbackup servers, I asked for install this command as soon as posible.

I use tar and the result was


/opt/openv/netbackup/bin/tar tvf /dev/nst0
Blocksize = 128 records
Hmm, this doesn't look like a tar archive.
Skipping to next file header...


Qrwx------ @@MaNgLeD.44/root;Domain Users@LAC 0 Oct 14 18:22 2005
--End header --
Lrwx------ 1/1                              272 Oct 14 13:40 2005 .SeCuRiTy.47
Erwx------ 1/1                               72 Oct 14 13:40 2005 @@MaNgLeD.45_Uname
--Extra header --
Trwx------ @@MaNgLeD.45/root;Domain Users@LAC 0 Oct 14 13:40 2005 /P/W7Mig-Bkp/LACARGBAL0021/C/BackUp *************/BKP/GPEREZ/P-list pag21.jpg
Lrwx------ 1/1                              272 Jan 30 15:07 2007 .SeCuRiTy.48
Erwx------ 1/1                               72 Jan 30 15:07 2007 @@MaNgLeD.46_Uname
--Extra header --
Trwx------ @@MaNgLeD.46/root;Domain Users@LAC 0 Jan 30 15:07 2007 /P/W7Mig-Bkp/LACARGBAL0021/C/BackUp ***********/BKP/GPEREZ/Tampa expenses detail.xls
Skipping to next file header...


Qrwx------ @@MaNgLeD.46/root;Domain Users@LAC 0 Jan 30 15:07 2007
--End header --
L-w------- 1/1                              272 Mar 11 17:40 2008 .SeCuRiTy.49
E-w------- 1/1                               72 Mar 11 17:40 2008 @@MaNgLeD.47_Uname
--Extra header --
T-w------- @@MaNgLeD.47/root;Domain Users@LAC 0 Mar 11 17:40 2008 /P/W7Mig-Bkp/LACARGBAL0021/****************/BKP/GPEREZ/Thumbs.db
Skipping to next file header...


Q-w------- @@MaNgLeD.47/root;Domain Users@LAC 0 Mar 11 17:40 2008
--End header --
Lrwx------ 1/1                              272 Aug  1 13:44 2005 .SeCuRiTy.50
Erwx------ 1/1                               72 Aug  1 13:44 2005 @@MaNgLeD.48_Uname
--Extra header --
Trwx------ @@MaNgLeD.48/root;Domain Users@LAC 0 Aug  1 13:44 2005 /P/W7Mig-Bkp/LACARGBAL0021/C/****************/BKP/GPEREZ/UNTITLED.PPT
[Prod root @ lacarglxap006 ~]

I use **** for confidentials rules. There is a employed name .

 

Cruisen


is not posible to downgrade, its a new completely infrastructure.

 

Any other idea?


Thanks

 

 

areznik
Level 5

Any encryption used on the tape when it was written? Any encryption set up on the drives reading the tape?

mph999
Level 6
Employee Accredited

Without the tape media header (which is missing) this tape will never import into NBU.

If the VTL made an exact copy of the tape, then although this was made 'outside' NBU, it should be possible to import it on a system that does not already have a record of a tape with the barcode/ media ID of the copied tape.  however, it would need to be an exact copy.

Here, either (1) the VTL didn't copy the tape correctly (missing media header) or, (2) the original tape was missing the media header for possible reasons I hace mentioned.

(1) - We cannot do anything about from a NBU side

(2) - Check the original copy of the tape (scsi_command -map will be fine).   Can you see a media header.

cruisen
Level 6
Partner Accredited

Hello Wolaf,

you have to go for

 cat SIZE_DATA_BUFFERS
65536

because by using different hardware devices this is the only setting you can be sure that will be understood.

Just modify and test, I hope this will change the behaviour of NBU.

Best regards,

Cruisen

Wolaf25
Level 4

Hi

If we can't import these tapes with NBU only, I have a big problem... and I will have a long discussion with EMC...

They offers to us Data Domain for replace the VTLs , explain to us that the operation is similar...

As a last try I will change SIZE_DATA_BUFFERS.. I hope it work...

I will keep informing to you guys.

Thanks

 

Wolaf25
Level 4

Hi, other questions guys

Can I restore using tar ?

Thanks

 

mph999
Level 6
Employee Accredited

SIZE_DATA_BUFFERS will not resolve this issue I'm afraid ...  Although it is a good idea to have this set to a 'sensible' value, it only affects performance.  262144 works well in general, along with NUMBER_DATA_BUFFERS set to 128 or 256.

tar should work:

 

(You can just repeatedly run /usr/openv/netbackup/bin/tar -xvf <device>, the notes below just show where you are on the tape.  You could also use mt to position to the correct 'file'/ 'record' if you wish or just let tar skip over them).

There was a diagram to go with this previous TN I wrote ages ago (internal only), it's a bit difficult to upload it here, so hopefully this will do

MH TM BH DATA TM BH DATA TM EH TM

 

 MH = NetBackup Media Header / Tape Label

This is used to identify the physical tape.  It contains the NetBackup Media ID, as generated from the barcode of the tape,  submitted to Netbackup via the robot firmware.

TM = TapeMarkhis is effectively 'MTWEOF ioctl' and is used to signify the end of a file.

BH = NetBackup Backup Header

This is the Netbackup header written prior to the 'data'.  It contains, for example the backupid.

DATA = tar archive

This is the actual data contained within the backup in tar format

EH = NetBackup Empty Header

This NetBackup Empty header is used to signify the end-of-data on a tape.  It is not the same as the logical end-of-data which would be represented by two TapeMarks, in the case of 1/2" Media (for example LTO media).  It may be worth noting that logical EOM is represented by a single TapeMark on certain media formats, for example 4 mm.


It is very easy to read through a tape using the /usr/openv/netbackup/bin/tar command.  The below is a description of events starting at the beginning of the 4 mm tape (mt -f /dev/rmt/xcbn rew) and repeatedly running tar -tvf </dev/rmt/xcbn> until an I/O error is obtained (Logical-End-of-Media).  It is not possible to use this method to read data from a multiplexed backup.

Position 1

Tape position File no=0 Block no=0:

This is positioned to the left of the Media Header, running /usr/openv/netbackup/bin/tar -tvf </dev/rmt/xcbn> produces ;

/usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

Blocksize = 2 records

Hmm, this doesn't look like a tar archive.

Skipping to next file header...

(This is the output given by /usr/openv/netbackup/bin/tar command when it encounters a NetBackup Media, Backup or Empty header).

Position 2

Tape position File no=0 Block no=1

This is positioned between the Netbackup Media Header and the TapeMark.

 /usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

no data in archive

(This is the output given by /usr/openv/netbackup/bin/tar command when it encounters a TapeMark)

Position 3

Tape position File no=1 Block no=0

This is positioned to the left of the 1st Netbackup Backup header

/usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

Blocksize = 2 records

Hmm, this doesn't look like a tar archive.

Skipping to next file header...

Position 4

Tape position File no=1 Block no=1

This is positioned to the left to the (1st) Data tar archive.

root@womble opscenter $ /usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

Blocksize = 64 records

drwxr-xr-x root/root  0 Jan 24 13:02 2011 /

drwxr-xr-x root/root  0 Jan 24 09:16 2011 /netbackup/

drwxr-xr-x root/root  0 Dec 15 22:54 2010 /netbackup/testdata/

-r--r--r-- root/root 400 Nov 29 09:53 2010 /netbackup/testdata/newfile1

-r--r--r-- root/root 400 Nov 29 09:53 2010 /netbackup/testdata/newfile2

(... output truncated for clarity )

Position 5

Tape position File no=1 Block no=2

This is positioned after the first data archive and before the TapeMark.

/usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

no data in archive
Position 6

Tape position File no=2 Block no=0

This is positioned after the TapeMark, but before the second NetBackup Backup header.

/usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

Blocksize = 2 records

Hmm, this doesn't look like a tar archive.

Skipping to next file header...

Position 7

Tape position File no=2 Block no=1


This is positioned at the beginning of the 2nd tar data archive.

/usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

Blocksize = 64 records

drwxr-xr-x root/root  0 Jan 24 13:02 2011 /

drwxr-xr-x root/root  0 Jan 24 09:16 2011 /netbackup/

 drwxr-xr-x root/root  0 Jan 25 13:25 2011 /netbackup/testdata/

 -r--r--r-- root/root 400 Nov 29 09:53 2010 /netbackup/testdata/file1

 -r--r--r-- root/root 400 Nov 29 09:53 2010 /netbackup/testdata/file2

 -r--r--r-- root/root 400 Nov 29 09:54 2010 /netbackup/testdata/file3

Position 8

Tape position File no=2 Block no=2

This is positioned after the second tar data archive, before the TapeMark

 /usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

 no data in archive

Position 9

Tape position File no=3 Block no=0

This is positioned before the NetBackup Empty tape header

 /usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

 Blocksize = 2 records

 Hmm, this doesn't look like a tar archive.

 Skipping to next file header...


Position 10

This is positioned after the Netbackup Empty tape header

Tape position File no=3 Block no=1

/usr/openv/netbackup/bin/tar -tvf /dev/rmt/0cbn

no data in archive


At this point, the tape drive position is reported as File no=4 Block no=0 and any further tar command results in an I/O error as we have reached the Logical end-of-tape.

 

mph999
Level 6
Employee Accredited

... also I see you mention the new master has different blocksize (262144).

This makes no difference, once the tape is written using blocksize X, it WILL be read using blocksize X, it cannot be changed.  There is a setting NUMBER_DATA_BUFFERS_RESTORE for changing the number of buffers for a restore, but there is no corresponding file for the size of buffers used in a restore, despite people posting this up in various posts, there is no such thing.  SIZE_DATA_BUFFERS only affects backups,

As I mentioned, the buffer size will not provide a resolution.

The questiosn are :

1. Does the original tape have a media header (it's possible it doesn't and that the VTL only copied what was there and some other issue caused the hmedia header to be missing.

2. If the original tape was good, why has the VTL lost the first 1024 bytes (the media header is 1K (in fact, all NBU headers are 1K (media header/ backup header and empty header).

mph999
Level 6
Employee Accredited

Capture_1.JPG

 

This is a slide from a presentaion I wrote recently.  It shows how the 'files/ records' on a tape are laid out, so is a visual representation of what is seen in scsi_command -map, which I thought you mighht find useful.

Wolaf25
Level 4

Martin

A lots of thanks, I will try to do.


I will keep informed.

 

 

Michael_G_Ander
Level 6
Certified

Way back the Netbackup tar had an option i for ignoring errors, very usefull when reading overwritten/damaged tapes manually.

Another option that is nice to know is the one for reading multiplexed backups, M back then

Think you can still get the options on unix/linux by doing tar --help

 

I am thinking that the first 64K block might be a VTL tape header

 

 

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

mph999
Level 6
Employee Accredited

Thats a good point Michael,a VTL tape header ... However ...

00000000: file 1: record 1: size 65536
01555929: file 1: eof after 1555929 records: 101969362944 bytes
01555930: file 2: record 1: size 1024: NBU BACKUP header
          backup_id lacargbaas099_1396049286: frag 2: file 2: copy 1
          expiration 2147483647: retention 9: block_size 65536
          flags 0x4: mpx_headers 0: resume_count 0: media CA1065
01555931: file 2: record 2: size 65536

Thare are '1555929' records, if you look at the picture I posted above you can see that each block of 'data' is a record, so in fact, there are 64k x 1555929 'blocks' of data on the tape,-before the next media header. which is aroung 100GB - thats' one real big VTL header ...

It's possible of course that the very first blcok is a VTL header that splatted the NBU media header, but I would have thought that the VTL header (if there is one on the 'readable' part of the 'tape' was followed by a filemark, and thus it would be one file/ record, and then the data started as a new file with multiple records.

This is looking more and more like the tape was over written (the old - External Event has caused SCSI rewind) which is beyond our control of coourse,  However Wolaf is very good technically I think, and I suspect he would have spotted this.  We can stop it, we can't detect it as such - but after a write, NBU checks that the position of the tape is where it should be (we wrote 100 blocks so the tape position should be at position 100 ) and so an overwriten tape should show up that way.

The other possibility is of course, that the tape was written by something outside of NBU.

It is, I'm afraid a mystry at the moment, perhaps if the data is readable by tar we will find out more.

 

Michael_G_Ander
Level 6
Certified

Was actually thinking of only the first 64K in file 1 record 1 as I had understood the EDL4100 VTL had written the tapes.

 

 

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue