cancel
Showing results for 
Search instead for 
Did you mean: 

vault duplication jobs

tinsoldier66
Level 3

Hi

I am running duplication jobs from virtual to physical tape as part of my daily Vault process.

I want to know where the size of these duplication jobs is set so I can make them smaller to make more use of the tape drives available. The whole process is currently taking too long as the duplication job sizes are so large.

Thanks

Andy

1 ACCEPTED SOLUTION

Accepted Solutions

RonCaplinger
Level 6

"...as the drives were often maxed out..."

Do you meant all 18 were in use at once?  Adding more tape drives to a single server will not speed up the backups, you would need to add more media servers and distribute tape drives among them.  I suspect you have very low throughput speeds if you really only have one master/media server.

The backup data comes from the client to the media server and then out to the tape drive.  Assuming fiber channel HBA's, you will likely have either 4Gbit or 8Gbit data path in your HBA's.  The combinations of NIC speeds, SAN speeds, PCI bus speeds, etc. plus overhead make a single media server too slow to stream data to more than about two physical tape drives fast enough to keep your tape drives moving without having to stop, wait for more data, reposition the tape to the correct block to write, write data in the buffer, stop, etc.  This "shoe-shine" effect just wears your tape drives down.

VTLs are different because they don't need to worry about tape repositioning during backups.  But you are still throttling your data through a small pipe that must also serve more than two physical tape drives, plus all the data coming into the VTL during backups.

In my opinion, without even seeing your policies or other configs, this is why your duplications are taking too long.  You need to increase the amount of data you can push from the VTL to the tape drives simultaneously.  That can only be done by adding media servers and moving some of the throughput load off of the current server.

View solution in original post

11 REPLIES 11

NBU35
Level 6

Size of duplication job can modified only in SLP.

In case of Vault it is not possible to manage the size of duplication job. however in vault profile you can change the creteria to select images for duplication & you can break a vault profile to multiple vault profiles, which will select small batch of images.

kindly correct me if I am wrong.

mph999
Level 6
Employee Accredited

The size of the jobs cannot be changed, it's 'hard coded' .  I've had a search for any 'touch file' tuning, but it doesn't appear there are any that are related to batching.

 

 

 

mph999
Level 6
Employee Accredited

Apart from Champ35s suggestion, is it the case that when the duplications are running as there are now, that they are running at a good speed, as in MB/s

As you explain they are taking too long, and if this is not down to a low copy rate, then is it the case you have simply reached the limit of what you can copy in a given time period, due to the number of tape drives available ?

 

Marianne
Level 6
Partner    VIP    Accredited Certified

We need to know more about your Vault config.

How many media servers?
How many VTL and physical drives atached to each?
How many 'write drives' in Vault Profile(s)?

Have you had a look yet at recommendations in NetBackup Vault Operator's Guide ?

For example: About increasing duplication throughput section...

tinsoldier66
Level 3

Thanks for the replies so far. they aren't running particularly slowly it's just that some are so large that they are still running after 24 hours when many others have finished and freed up drives that could be used.

Last night for example we have a couple that are 1.8TB in size and one that is still running is 7.8TB and has currently written 4.3TB in 23 hours!  While a lot of others are tiny in comparison.

I'm not sure why some duplication jobs are bigger than others. Ideally we'd break the larger ones down so the drives are fully utilised for the duration of the process.

 

Marianne

1 x master/media server - 56 VTL drives and 24 Physical drives

(only 22 are used for duplication, 2 are reserved for another process)

 

Not checked that Guide but will have a peruse Thanks :)

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Check this out

http://www.symantec.com/docs/TECH169548

Read right to the end (it might not seem relevant but you;ll see the last bit is)

mph999
Level 6
Employee Accredited

Not seen that TN, clever ...

watsons
Level 6

Great tecnote you found there, Riaan!

For tinsoldier66, vault does not set the size of duplication, it chooses from what you set in vault profile : the date range, disk or media volume pool, retention, policy, schedule etc. , so if you know there were lots of full backup last weekend, and you're going to duplicate them via vault, some images will surely be large. If you want to exclude those images for now, just filter them out using policy/schedule in attributes and leave it for latter duplication.

Marianne
Level 6
Partner    VIP    Accredited Certified

Just saw this:

1 x master/media server - 56 VTL drives and 24 Physical drives

There is no way that a single server can handle this amount of tape drives.

Most servers will battle to keep more than 2 tape drives streaming.

If you really need that many tape drives, you need to add a couple of media servers. 

tinsoldier66
Level 3

hi Marianne

Do you mean the number of Virtual or Physical drives? Or both?

We originally had 50 VTL and 18 physical but we had an opportunity to add more so thought that would help, particularly with the duplication process as the drives were often maxed out and thought that if we added more it would speed up the process. WE use 22 drives for duplication and keep 2 free and they are all in use throughout but what I couldn't fathom is why Netbackup would lump over 10TB of data together in one duplication job when you had lots of smaller jobs of only 500gb or less.

The overnight backups which use the VTL seem to work fine on the 56 drives.

Regards

Andy

RonCaplinger
Level 6

"...as the drives were often maxed out..."

Do you meant all 18 were in use at once?  Adding more tape drives to a single server will not speed up the backups, you would need to add more media servers and distribute tape drives among them.  I suspect you have very low throughput speeds if you really only have one master/media server.

The backup data comes from the client to the media server and then out to the tape drive.  Assuming fiber channel HBA's, you will likely have either 4Gbit or 8Gbit data path in your HBA's.  The combinations of NIC speeds, SAN speeds, PCI bus speeds, etc. plus overhead make a single media server too slow to stream data to more than about two physical tape drives fast enough to keep your tape drives moving without having to stop, wait for more data, reposition the tape to the correct block to write, write data in the buffer, stop, etc.  This "shoe-shine" effect just wears your tape drives down.

VTLs are different because they don't need to worry about tape repositioning during backups.  But you are still throttling your data through a small pipe that must also serve more than two physical tape drives, plus all the data coming into the VTL during backups.

In my opinion, without even seeing your policies or other configs, this is why your duplications are taking too long.  You need to increase the amount of data you can push from the VTL to the tape drives simultaneously.  That can only be done by adding media servers and moving some of the throughput load off of the current server.