Duplication Backlog

[arctics2]

Hello,

We are in the process of implementing 2 Netbackup 5230 appliances.  The plan is to use SLP's with AIR to replicate backups to the remote appliance and to duplicate to tape.  The problem is that the tape library is delayed for about a month.  If we implement, without the tape library from day one, will we have an issue duplicating completed backups to tape?  

[RamNagalla]

hi,

you would probably need to do the  manual duplication to tape for the images that are replicated to remote appliance.. that were happen before tape library deployment.

manual duplication only can do for the SLP contoroled images.. once the SLP compleated its set operation or the images has been cancled from the SLP.

and once you deploy the tape library add the 3rd Operation in the SLP that is tape duplication.

[Marianne]

As per Nagalla's post, you will have to duplicate images manually. Either use the GUI or in a script using bpduplicate.

Duplication to tape can be added to SLPs when hardware has been installed and configured. New backups will be duplicated to robot from that point onwards.

[rk1074]

Netbackup 7.6 has introduced a duplication window for the SLP where in you have an option to delay the duplication till the time images are about to expire... so you can start the duplication of you backups ,lets say, 4 hours before the image expiration.

Maybe you can use this feature if you can afford to upgrade you NBU env.

[Nicolai]

You will need the robot in place to configure the storage unit that the SLP would use for duplication. Without robot no STU ....

1 month of backlog would properbly take weeks to complete.

[DrR]

I agree with Nicolai - it could take a long time to complete but on the other hand it really depends on how big your tape library is and how much data you back up.  I run two LTO4 drives and my weekly full backup of around 10 TB finishes mid-day Monday.  However I don’t bother writing to tape my daily backups as I have enough space on the disk for their lifecycle so I have time to catch up.
The danger is that the tape library will not catch up and your disk pool will fill up before the old copies expire.  This will be down to your design.  You should have some guestimate of how long your normal duplication to tape will take.  If you have the disk space and expect you tape library to be idle most of the time it should eventually get there.

You might consider that not all of you backups form that one month period have to go to tape?

If you do this I would recommend doing them in batches and consider the priority of the duplication jobs against ongoing backups. I also group mine by expiration date.
I normally make the tape the primary copy when duplicating – makes it easier to track which image is ready to be manually expired.

[arctics2]

Thanks for the comments.  We've got about 15 TB to write to tape, and will be running 4 x LTO6 drives.

The problem is that the requirement is that ALL production full AND incremental backups must be duplicated to tape (I know, this is a whole separate discussion on it's own, but for now, this is the requirement).  

I'm not too concerned about running out of space on disk as the storage is a farily large dedupe store, but more concerned about the time it will take for the months worth of backups to duplicate to tape.

[EnriquePereira]

I do not think you will have issues. As the appliances are new, there will be little fragmentation on the pool and rehydratation should work at good speed. 

You probably could get something between 100 and 200MB/seg duplicating to tape, so 15 TB will take between 1 and 2 days (of course, this is only a estimation, please tell when you run it what is the real speed). Maybe during duplication to tape could help to suspend the source duplications (not the backups, just the air duplication).

As a trick, you can select convert to primary copy the copy that goes to tape. This can be a easy way to track wich images you have left on the msdp pool to duplicate to tape (the primary ones are still not replicated). Later you can convert again the MSDP image to primary when you finish the tape out.

[mph999 · Accepted solution]

... but don;t forget, the duplications have to happen while the drives also cope with the regular day-to-day stuff ...

Without knowing exact details it's impossible to make any accurate comments.

Based on what I have seen in the past ...

What usually happens, is that SLP is an after thought and many systems do not have sufficient throughput to cope with SLP, hence the big demand for SLP windows (which arrived in 7.6).

(Not saying that SLP shouldn't have had windows in the first place ... )

Anyhow, when we see issues with cases were for whatever reason, an SLP backlog has built up, sometimes for even just a few days, if the system only just has enough capability to cope with the normal duplications each day, it can be impossible to catch up.

So it comes down to how much idle time the drives have that can be used to tackle the backlog.  I would also consider that appllicable tuning might need to be applied in various areas to cope with the extra demand a months worth of backlog will place on the system (nbrb, emm etc ...).

[DrR]

I think you need to work out the size of your incremental.  The weeklies should catch up but if your daily incremental backups are large that would be a concern.  My weeklies are 10 TB but my dailies are only 1 TB as the data rate of change is low.  This would also depend on if you are doing cumulative or differentials.

Impact on you system might affect throughput of you backups - I don’t notice it with mine but my tape library is considerably slower. 

I also normally throttle the max number of partly full tapes per set quite low to be conservative on tape usage (so I don’t exceed my eject ports max number of tapes)  But in times like this would you may want to leave it high and put up with a few more partly filled volumes -> extra tape -> extra storage.

[arctics2]

Thanks Guys.  We did some quick calculations, and by the looks of it, the backlog would grow, and could potentially take a very long time to catch up on.  Since it's a new implementation, we've decided to hold off on implementing for as long as we can, and hopefull the tape library will arrive sooner rather than later.