Help with backup strategy

[Gene_Magerr_2]

I have a dell 136T autoloader. It will backup 28 Terabytes uncompressed. It holds 72 LTO3 tapes. It is fully loaded with 6 drives and a 4 Gig network pipe. Right now I'm backing up around 15 to 20 terabytes throughout the week. I have the jobs doing one full backup a week with differentials throughout the rest of the week. The full backups of these jobs are staggered throughout the week. Here's where I need help. I'm wanting to keep 2 months of backups that will allow us to retrieve from any day from those 2 months. Then anything older than that I'd like to be able to retrieve from the last weeks full backup? I'm not sure. How can I accomplish this? It's a huge amount of data and I'd like to come up with a more comprehensive backup strategy. I'm burning through tapes like crazy and these tapes are not cheap. $10,000 for 100 tapes. Can someone help? thanks in advance.

[ray_littlefie1]

Can you clarify your requirement? In one sentence you seem to indicate you want the library to retain 2 months worth of backups. then the next sentence-

"Then anything older than that I'd like to be able to retrieve from the last weeks full backup? "

-you lost me?

Are you saying you want to retain 60 dyas worth of data in the library or are you saying you want backups of data that is less than 60 days old? Or is there an offsite rotation that you did not mention?

[Gene_Magerr_2]

Sorry I wasn't clear. I really don't know the correct way to say this. I think it will be an offline rotation strategy. I'd like to have 2 months worth of tapes. From those tapes I'd like to be able to retrieve data from any day of any week. In order to do that I'm assuming I'd load the tape device, run my backups for a week and on monday pull those tapes and put them in a box called week one. I'd do the same for the last 3 weeks of the month. So i'd have 2 months (8 boxes of tapes marked Jan week 1,2,3,4 - Feb week 1,2,3,4) So now when March comes around Would I take January Week 4 and put it in a safe? Would i put January weeks 1 - 3 back into rotation so I could overwrite them? Since I'm doing full backups through the week with differentials am I going to be able to recover data from the last weeks tape set? or is it going to ask me to put a tape in from one of the previous weeks? Is this clear? Should I just run full backups on the last week of every month and not do the differentials? Can you recommend a better way to accomplish this? Thanks so much for the response.

[ray_littlefie1]

Please do this:

Go to Reports -> Operations Overview, select a range of 30 days and run the report. Take the total size of Backups number and divide by 30 then drivide by 1024.

This will provide the avg. daily backup amount (regardless of (full/differential)

Also, how many Active media sets are you using for your backups (hopefully not too many, as this will reduce tape utilization efficency). What are the OPP and append settings currently?

[Gene_Magerr_2]

Ok I ran the report and got
39046214
divides that by 30 and got
1301540.4666666666666666666666667
divided that by 1024 and got 1271.0356119791666666666666666667

I'm using one media set, (the default media set 1)
I have the media set to
Inifinate - Don't allow overwrite
Infinate - Allow Append
I put the tapes in, they go into the scratch media I inventory them and the jobs pull the tapes from the scratch media. I'm writing to the tapes until they reach 0 capacity then taking them out of the autoloader and putting them into a safe. I know there has go to be a better way to do this. I'm anxious to hear your suggestions. Thanks.

[ray_littlefie1]

So on average you backup 1271GB/Day. If you assume 500GB/tape (~1.2:1 compression) then you consume 2.5 tapes a day.

To keep a 60-day supply with your current settings it would seem you need 150 tapes minimum (2.5X60). Because you have four tape drives; they're likely all busy doing individual jobs each night. So, there is some job append inefficency and I'm sure there are some tapes that are not being used to full capacity. I would factor in 1 tape per night for this and add 60 tapes and assume you currently need 210 tapes to get your 60 day retention level.


Obviously you need to rotate tapes- but is your biggest concern the amount of tapes you're using or is it the time required to remove and insert tapes?




If it was me, I would want a simple rotation scheme that can be easily explained and maintained even if it cost a few more tapes than other methods.
I would partition the library into 2 sections: one section would be 20 slots and would be for Full backups only. 20 slots (10TB) should provide enough space for one each weekly full backup. I would create 8 sets of 20 tapes (160 total) and create a 2 month rotation scheme with these sets.
The remaining 54 slots would be for daily incremental (I don't think you can afford to do differential with what you have). These tapes would never leave the library but rather age out and get reused at a 60 day OPP. This scheme uses (160+54) at least 214 tapes.
You would need to change your media set settings to an OPP=60days, infinite append. You would need to create a library partition within veritas, and would need to define/re-define once/week full backup jobs to go to the 20-slot partition, and daily incremental jobs to go to the 54 slot partition.

Once each week, you would remove all of the tapes from the 20 slot partition and replace them with the next rotational set.

Again, this is based on some loose calculations and may not work as well when put into practice. Other methods would be to stop doing weekly full backups, and do only one or two Full backups per month and then the rest of the month do differentials or incrementals. (note: never mix differential and incrementals inbetween FUll backups when using the clear archive bit method). This would require less tapes, but it depends on how much your data changes each night.

If the time comes around when your budgeting for a new library, take the total amount of disk space you plan to backup and multiply by 10, and that would be a starting point for library capacity. For your situation, somewhere around a 200-250 slot library would have been nice. ;)

[Gene_Magerr_2]

Ray
This sounds like a good plan. Do you think maybe you can walk me through this process? Partitioning the device and creating the sets etc. my email address is gmagerr@mgae.com I'm setting up a new server to take over as the new tape server. i will probably just recreate all of the jobs I have. This would be a perfect oppertunity to do this. I really appreciate all of your help. Thanks.

[Ajit_Kulkarni]

Hello Gene,


Can you please update on the issue ?


Regards


NOTE : If we do not receive your reply within two business days, this post would be marked "assumed answered" and would be moved to "answered questions" pool.

[Gene_Magerr_2]

Thanks Ray
I think partitioning the slots is a good idea. My concerns are tape cost and being able to go back and restore something when I need to. I don't think incremental backups are going to work because a good deal of the things i backup are databases. I think you can only do differentials on sql databases. I have a couple of questions on this strategy. Say I partition 20 slots for the full backups and i discover i don't need 20 or i need more than 20. I'd have my full backup jobs pointing at those partitions. If i did need to expand the partition set would I have to go through all of the backup jobs and re-select the partition sets, or would the jobs just use the newly created partitions. The other day i had to restore something and Veritas alerted me to put in one of the tapes from a previous backup. I did, then it asked me to put in a second tape. When I have to restore something from one of the full backup sets is it going to ask me for other tapes or is everything I need going to be on that tape. I'm asking because if I'm overwriting tapes outside of the 20 slots I assign for full backups I want to feel confident everything is going to be on those full backup tapes and it's not going to ask me to insert one of the tapes I overwrote. the last question. I've been doing full weekly backups with differentials throughout the week for a few months now. Is there a relatively easy way for me to go through all of the tapes I've already written to, locate and pull the full backups out? So for example I'd like to pull out four weeks of full backups so I have all the full backups for the four weeks of last month? Thanks again for all of your help. It is greatly appreciated.

[ray_littlefie1]

Regarding re-partioning the slots, the device database is all part of the ADAMM MSDE database and will dynamically update all of the jobs. However, I have experienced times when all looks well after a re-partitioning change, and then the jobs fail. I then had to go in and re-select the partition for each job. Using backup policies reduces the administrative workload on such things.

Regarding restores and data spanning multiple tapes, in the restore section when you select the top level of your restore selection, the preview pane will indicate the Media labels required to perform the entire restore. This is the media family, and none of these tapes can be individually overwritten without endangering all of the data on all of the tapes.(actaully it''s fine as long as the on-disk catalogs remain intact). In your proposed scenario, the OPP is the same for the fulls as it is for the differentials. So, I would not anticipate any data being lost from the tapes that remain in the library as opposed to the tapes that get rotated. The purpose of the rotation in your case is to compensate for insufficent slots.

There is not a real easy way to determine Full's from differentials with your current data. You only have one media set, so it is likely the Full's and differential are mixed on the same tapes. You can accomplish full vs. differential backup seperation using two media sets "full & "differential", and then redefine the backup jobs to go to the appropriate media sets. This will lead to some tape usage inefficiency.

[Gene_Magerr_2]

> Regarding re-partioning the slots, the device
> database is all part of the ADAMM MSDE database and
> will dynamically update all of the jobs. However, I
> have experienced times when all looks well after a
> re-partitioning change, and then the jobs fail. I
> then had to go in and re-select the partition for
> each job. Using backup policies reduces the
> administrative workload on such things.
>
> Regarding restores and data spanning multiple tapes,
> in the restore section when you select the top level
> of your restore selection, the preview pane will
> indicate the Media labels required to perform the
> entire restore. This is the media family, and none of
> these tapes can be individually overwritten without
> endangering all of the data on all of the
> tapes.(actaully it''s fine as long as the on-disk
> catalogs remain intact). In your proposed scenario,
> the OPP is the same for the fulls as it is for the
> differentials. So, I would not anticipate any data
> being lost from the tapes that remain in the library
> as opposed to the tapes that get rotated. The purpose
> of the rotation in your case is to compensate for
> insufficent slots.
>
> There is not a real easy way to determine Full's from
> differentials with your current data. You only have
> one media set, so it is likely the Full's and
> differential are mixed on the same tapes. You can
> accomplish full vs. differential backup seperation
> using two media sets "full & "differential", and then
> redefine the backup jobs to go to the appropriate
> media sets. This will lead to some tape usage
> inefficiency.

Thanks Ray
I was under the impression that partitioning the slots would do what you suggested doing with the media sets full and differential. If I have the weekly jobs going exclusively to the full backup partitions I'm still going to have data written to the other tapes in the differential partition? is that correct? Thanks again Ray.

[ray_littlefie1]

I was trying to illustrate a difference between making two media sets, or making two library partitions.

Creating the library partitions makes a physical division in what tapes can be used. Creating media sets makes a logical partition in what tapes can be used.

I suggested the library partition because you wanted to physically move tapes in/out of the library in your initial thread. The comment I made about the two media sets was meant to be a hypothetical "if you would have set it this way, you could now identify full backups from differentials. -Sorry for the confusion.


In either case, the jobs need to be modified so that data gets sent to the correct partition.

So in your case, you should have jobs that are full backups going to device "LibraryName" and differentail backups going to device "LibraryName", or something like that.

[Gene_Magerr_2]

Thank you Ray I am going to set up the new server today. Just so I completely understand, When I partition the slots, all of the full backups go to slots 1 thru 20. I will remove these tapes each week (4 sets for each month) these tapes will have all the data to restore what was backed up to them. In other words I will not be needing the differential tapes to restore from these tapes. Correct? Thanks again you are extremely helpful.

[Amruta_Bhide]

Hello,

We apologise for the delay in responding. Could you please Update us on the latest State of the issue?

******************************************************************
*****************************************************************

Note : If we do not receive your reply within two business days, this post would be marked �assumed answered� and would be moved to �answered questions� pool.


Thanks.