11-17-2015 06:06 AM
Hi All,
I ran duplicates using bpduplicate command using BID file.
I was looking at activity monitor logs. Please tell me what does this mean -
What is happening at "positioning to file"?
It does not come when we run duplicates manually one by one from Netbackup Administration console --->catalog.
Thanks.
Solved! Go to Solution.
11-17-2015 06:57 AM
See this blog
https://www-secure.symantec.com/connect/blogs/understanding-how-netbackup-writes-tape
Each "file" is a netbackup fragment with the fragment (file) size configured in the storage unit definition.
If you think of a old cassette music tape, with your favorite as song 3. You then has to fast forward 2 songs to hear it or fast fordward to the end to record a new performance.
Netbackup work in the same way. It can fast forward before listing (reading) or fast forward to the end, to record a new song (backup).
If you want to copy a lot of song (backups), to a new tape, you may have to fast forward to get song 2, 6, 12 and 24. The diffrence here is that NBU actual tells you where it is on tape and what its doing right now.
The messages are informational only, but can be used to find performance issue during write operations.
11-17-2015 06:07 AM
Forgot to attach log.
Attaching log.
11-17-2015 06:57 AM
See this blog
https://www-secure.symantec.com/connect/blogs/understanding-how-netbackup-writes-tape
Each "file" is a netbackup fragment with the fragment (file) size configured in the storage unit definition.
If you think of a old cassette music tape, with your favorite as song 3. You then has to fast forward 2 songs to hear it or fast fordward to the end to record a new performance.
Netbackup work in the same way. It can fast forward before listing (reading) or fast forward to the end, to record a new song (backup).
If you want to copy a lot of song (backups), to a new tape, you may have to fast forward to get song 2, 6, 12 and 24. The diffrence here is that NBU actual tells you where it is on tape and what its doing right now.
The messages are informational only, but can be used to find performance issue during write operations.
11-20-2015 05:24 AM
Hi Nicolai,
Thanks for such a good example.
I saw the article that you have provided above.
Its a good article. But I want to ask one thing.
How netbakup handles this -
BOT|server1part1|server2part1|server1part2|server2part2|EOT
Now suppose the image server2part2 is expired (the last one expired) i.e. blankspace is created, other images are yet to be expired.
Will the next backup use the tape and wite it there where the last image expired since it writes linearly and there is blank space created after expiration of last image.
Please suggest.
11-20-2015 05:30 AM
No!
NetBackup will not write to any "space" on a tape UNTIL the very last image on tape has expired and the tape has been deassigned.
NetBackup does not remove the image from tape, it just removes the image from the catalog.
11-20-2015 05:40 AM
Hi revarooo,
when this is the case it writes to the blank space -
BOT|server1part1|server2part1|server1part2|Blank space|EOT
When this is the case it nt writes on the tape according to you)-
BOT|server1part1|server2part1|server1part2|expired image|EOT
That means netbackup is not treating the last expired space as blank space, right?
11-21-2015 01:28 AM
I think Netbackup will start writing at the last valid (valid is important here) image on tape - that is the end of "server1part2".
But pratical this will never happen, because Netbackup doesn't mix retention. The oldest backup will always be in front and the newest always in the end.
I can think of two scenarios where Netbackup would re-use space at end of tape:
BOT|server1part1|server2part1|server1part2|Non MPX failed Backup|EOT
11-21-2015 06:13 AM
01-05-2016 05:59 AM
Hi Nicolai,
That means when this is the case -
BOT|server1part1|server2part1|server1part2|expired image|Blank space|EOT
it will write the new backup images at "expired image" space (if it is in the end of the tape or before blank space) and then proceed for Blank Space if any.
Did I understand right?
01-06-2016 12:55 AM
yes - that is my understanding (please note: I could be wrong) - but the expired image must be non-mpx else the next image will start at blank space.
01-06-2016 03:49 AM
Seems Nicolai is correct ...
At the end of a backup, either mpx or non-mpx NBU writes an empty header :
root@womble 160106 $ scsi_command -f /dev/rmt/1cbn
Inquiry data: removable dev type 1h HP Ultrium 1-SCSI E38W
root@womble 160106 $ scsi_command -map -f /dev/rmt/1cbn
00000000: file 1: record 1: size 1024: NBU MEDIA header (A00001)
00000001: file 1: eof after 1 records: 1024 bytes
00000002: file 2: record 1: size 1024: NBU BACKUP header
backup_id womble_1451909069: frag 1: file 1: copy 1
expiration 1452513869: retention 0: block_size 262144
flags 0x0: mpx_headers 0: resume_count 0: media A00001
00000003: file 2: record 2: size 262144
00000004: file 2: record 3: size 98304
00000005: file 2: eof after 3 records: 361472 bytes
00000006: file 3: record 1: size 1024: NBU BACKUP header
backup_id womble_1451909106: frag 1: file 2: copy 1
expiration 1452513906: retention 0: block_size 262144
flags 0x0: mpx_headers 0: resume_count 0: media A00001
00000007: file 3: record 2: size 262144
00000008: file 3: record 3: size 98304
00000009: file 3: eof after 3 records: 361472 bytes
00000010: file 4: record 1: size 1024: NBU EMPTY header (file 3) <<<<<<<< HERE IS THE EMPTY HEADER
00000011: file 4: eof after 1 records: 1024 bytes
eot
When a new backup is written to the tape, NBU searches for the Empty header, well, in fact it knows where the Empty header 'should' be , it then reads it to confirm it really is in the correct place. It then rewinds, overwrites the Empty header with a Filemark, and then writes the new backup header and continues writing the data. This can be clearly seen in the bptm log
11:35:08.567 [9715] <2> io_position_for_write: position media id A00001, copy 1, current number images = 2
11:35:08.567 [9715] <2> io_position_for_write: locating to absolute block number 10, copy 1
11:35:08.620 [9715] <2> io_position_for_write: locate block is done
11:35:08.567 [9715] <2> io_position_for_write: position media id A00001, copy 1, current number images = 2
11:35:08.567 [9715] <2> io_position_for_write: locating to absolute block number 10, copy 1
11:35:08.620 [9715] <2> io_position_for_write: locate block is done
11:35:08.649 [9715] <2> io_position_for_write: processing empty header, filenum = 3, bid = (empty_file), copy 1 <<< Found the empty header
11:35:08.649 [9715] <2> io_position_for_write: empty header found on A00001, OK, copy 1 <<<< Checks it really is the empty header
11:35:08.649 [9715] <2> io_ioctl: command (2)MTBSF 1 0x0 from (bptm.c.22814) on drive index 0 <<<< Rewinds to before the empty header
11:35:08.786 [9715] <2> io_ioctl: command (0)MTWEOF 1 0x0 from (bptm.c.22866) on drive index 0 <<<<< Overwrites the empty header with a file mark
I expired the last backup on the tape (womble_1452080100), and wrote a new one ( womble_1452080389 ) ...
So we have some 'free space' now on the tape ... before the empty header ...
Lookinf at scsi_command -map output ....
00000006: file 3: record 1: size 1024: NBU BACKUP header <<<<<<<< BACKUP HEADER OF IMAGE BEFORE womble_1452080389
backup_id womble_1451909106: frag 1: file 2: copy 1
expiration 1452513906: retention 0: block_size 262144
flags 0x0: mpx_headers 0: resume_count 0: media A00001
00000007: file 3: record 2: size 262144
00000008: file 3: record 3: size 98304
00000009: file 3: eof after 3 records: 361472 bytes
00000010: file 4: record 1: size 1024: NBU BACKUP header <<<<<< BACKUP HEADER FOR womble_1452080389
backup_id womble_1452080389: frag 1: file 3: copy 1
expiration 1452685189: retention 0: block_size 262144
flags 0x0: mpx_headers 0: resume_count 0: media A00001
00000011: file 4: record 2: size 262144
00000012: file 4: record 3: size 98304
00000013: file 4: eof after 3 records: 361472 bytes
00000014: file 5: record 1: size 1024: NBU EMPTY header (file 4)
00000015: file 5: eof after 1 records: 1024 bytes
eot
We see teh previous image on the tape is womble_1452080389, so indeed, it has overwritten the womble_1452080100 image which I expired.
This isn't possible for mpx images, as we don;t actually know where the data is for the image (it's all mixed in with other images)
01-06-2016 06:16 AM
Great. Thanks mph999 and Nicolai and others.