Small restorejob reads a lot

[Jens_Hyldedahl_]

Hi

I have been wondering for a while now, and I hope that somebody would be able to answer my question. I have just run a small restore job of a 250 Mb file, and the Job details window show that Current kilobytes written is 18,5 Gb even though it has only the written the 250 Mb file. Why does it read a lot more than needed?

JHR

[J_H_Is_gone]

I have not seen it anywhere but this is what I always thought.....

it starts counting up on a restore as soon as it starts reading the tape.

so I always figured that it was saying how much data on the tape it has read through (passed over) trying to find your file on the tape.

[Jens_Hyldedahl_]

I think it sounds weird that it has to traverse more than 18 Gb of data to be able to restore a single. Why isn't it able to just position the tape to that file? The job takes around 30 minutes to complete on a LTO3.

[cmumma]

What is the fragment size set to on the storage unit?  If you want it to find the file faster decrease the fragment size for the storage unit.  I usually set it to 4GB or so.

Message Edited by cmumma on 06-23-2008 08:35 AM

[J_H_Is_gone]

Well as I understand it, a restore can position direclty to the start of an image.... but still has to pass over all the other images on its way ( this is what I think it is showing).  Then once to the correct image it has to go find the start of that file.

Jens never said anything about it taking a long time, just wanted to know why the "written" showed so big on a restore.

this is only my opinion.... but seems to old true.... as I had the same question and saw that if the restore went fast ( file was near the start of the tape the number was low.... if the file was near the end.... the number was high)  (figured this about by running the report of images on tape and see where on the tape the image for my restore was in relation to other images on the tape).

[Jens_Hyldedahl_]

Thanks for the tip about reducing fragment size. I have set it to 8 Gb.

[Ron_Cohn · Accepted solution]

If you do not specify a "fragment size", then when trying to restore a single file, NetBackup has to scan the entire tape.  By specifying a "fragment size", this creates a "tape mark" and generates an entry in the catalog detailing which fragment number a file begins.

In our case, we use LTO-3 drives.  Without specifying a fragment size, it took approximately 1.5 hours to reach a file at the end of the tape.  By setting our fragment size to 10GB, the drive positioned to the correct segment in no time and reached the file at the end of that fragment in less than 3 minutes.

Specifying too small fragment size will somewhat slow down your backup as it has to create a "tape mark".  The performance and tuning guide gives a very good explanation of this.

[J_H_Is_gone]

can I ask why you would do a frag size instead of "Take Checkpoint"?

with a checkpoint it can position faster .... well at least my understanding of what I read.

[Ron_Cohn]

---

J.Hinchcliffe wrote:

can I ask why you would do a frag size instead of "Take Checkpoint"?

with a checkpoint it can position faster .... well at least my understanding of what I read.

---

Checkpoint applies to the time data is being written to the tape.  For example, if you have a long running backup and for some reason it fails, when restarting the job, it begins at the last checkpoint and continues forward from there.

Fragment Size helps you to locate a file faster for restoration purposes. 

[Andy_Welburn]

Checkpoint restart (Policy Attribute):

"Checkpoints during a backup are beneficial if a backup fails. Without Take checkpoints every enabled, a failed backup restarts from the beginning of the job. By taking checkpoints periodically during the backup, NetBackup can retry a failed backup from the beginning of the last checkpoint rather than restart the entire job. Note that checkpoints cannot occur while a file is backed up. Checkpoints are saved at file boundaries, and point to the next file in the list."

Fragment Size (Storage Unit Field Descriptor):

"Fragmenting multiplexed tape backups can expedite restores. Fragments allow NetBackup to skip to the specific fragment before searching for a file. Generally, NetBackup starts at the beginning of the multiplexed backup and reads tar headers until it finds the file.
...
The Reduce fragment size setting is intended primarily for storing large backup images on a disk type storage unit. Backups may be fragmented on disk to increase performance when the images that Storage Migrator has migrated are restored. For example, a restore is faster if a 500-megabyte backup is stored in 100-megabyte fragments. Storage Migrator needs to retrieve only the specific fragment with the file rather than the entire 500 megabytes."

So, one is beneficial for backups the other for restores.