Need suggestions for best backup practices for our environment

[Greg_Saunders]

We are trying to determine the best way we should be doing backups using 11D. Let's assume first that we are not going to use CDP (Continuous Data Protection). We have had some issues even getting this to work so let me start with having a system in place that does NOT use CDP.

First... the Media Server and current storage available:
1 Media Server 2 500GB drives with OS on the first 25GB of 1st 500GB drive.
2 NAS type servers for extra B2D needs... each have 850GB available.

Second... the servers needing to be backed up and the data sizes.
1 Exchange Server (Information Store) 80GB
1 SQL Server (Simply Data backup of SQL BAK files) 8GB
1 Large File Server 300GB
1 Citrix Server with data files 8GB
1 Web Server with data files 40GB

So basically you see our backup storage areas and our data needing backup.

In the past we would have created B2D Folders representing each Server. (ie. TBGEXCHANGE, TBGSQL, TBGDATA, TBGCITRIX, TBGWEB)
These B2D are placed on 1 of the 3 backup locations mentioned above and tried to pick locations so the largest data would have the largest drive.

For example the 300GB File Server data would go to 1 of the NAS storage devices that has 850GB available.

The Exchange would go to one of the local 500GB drives on the media server.

And the rest could just go where it best fit.

The one thing I need to happen is that I NEVER want to run out of disk space. There should be some way to ALWAYS make sure that when it reaches a certain point that it will start overwriting that B2D files so that it continues to rotate. I will have duplicate jobs that dump data at certain times to some large external drives that get taken offsite.

1) So how do we make sure a B2D never just fills up causing all backups for that location to cease???

2) Is there a way to make the B2D to always maintain the LAST x backups and when space is needed simply overwrite that oldest backup? Essentially we would like to ALWAYS have several backups available for a particular source at any given time. Hopefully you get the idea of what we want to happen in regards to B2D... and we are looking for even better ways of doing things. Ultimately we would like the least amount of interaction (other than making sure we get success alerts)

3) Is there a way to have Backup Exec save certain backups permanently like Yosemite does with their Grandfather, Father, Son rotation features?

4) If you just want to give your opinion on how you would setup backup needs for the above configuration please don't hesitate to share!

I know there are other questions, but I'll create separate posts for those as I think of them.

Thank you very much for your time!

[Bradley_Willads]

Greg,

The key for overwrite, it to make sure your retention period is setup properly. The longer your retention period, the more data will be stored.

Depending on how you target your jobs (to what B2D folder), I would create Media Sets that each device uses. B2D folder 1 has its own media set, B2d folder 2 has its own media set, etc.

Going on this scenario, where the Exchange goes to the 500GB B2D folder, you can store approx 6 backups. So your overwrite protection period should be no more than 6 days, otherwise you will run out of space.

I would follow that same rule for the rest of your jobs as well, to find out how much Overwrite protection is needed for each resource.

[Greg_Saunders]

You would think there would be some better features for controlling the number of backups held in a B2D Folder. So if I use the example of protect for 6 days I would I guess I set the job to overwrite and it would append for those 6 days. And then on the 7th day it would do an overwrite. NOW would that overwrite ALL of the last 6 backups or just start over with the oldest still leaving the last 5 intact as it goes.

It seems dangerous if it overwrites the whole thing without leaving them intact.

Oh... I do use a different media per b2d folder as you suggested... I found that to make the most sense.

Thanks for your help thus far... more ideas, thoughts, and opinions are greatly welcomed and appreciated!

[Bradley_Willads]

Greg,

If you perform infinate appends, and don't choose to overwrite your media, you could end up in a bigger mess than you started out with.

If you always append, every piece of media you use will then be part of that "family". That means if your server goes down, and you have to reinventory/catlog your media you may be asked for media that doesn't even exist, because Backup Exec konws they are part of the tape family. This is why its strongly recommended to start new families, and the only way to do that is by choose the option "Overwrite media", instead of "Append to media, overwrite if no appendable medai is available".

Second, if you have very large B2D files (like 500GB per sea, there is a good chance that will never be overwritten. When a backup is run, and it uses that single piece of media, the software isn't smart enough to overwrite the older sets within that file. It looks for additional media only.

Third, you can set it up such that you only have x amount of media in your B2D folder. To do this, it will take a little big of configuring and some math.

For example, we'll use your Exchnage backup for an example. You would need to configure your Backup To Disk folder to "Allocate the maximum size for backup-to-disk files". I would configure the maximum size for B2D files to ~80GB, the size of your database. This will then only create 6 - 80GB files for your backups.

However, it's not recommended to use that large of a file, only because Windows SMB has been known to have problems with very large files. But that is one scenario in which you can control how many files are created.

[Greg_Saunders]

Perhaps my understanding of how the B2D, overwrite protection, and what not is not as clear as I thought. I am so use to some other products that to leave less chance of confusion...

1) So let's say I set exchange media to have an overwrite protection of 6 days. Then what will happen is when the job is set for overwrite it will see the protection and simply append for up to 6 days. Then on the 7th day it will actually overwrite. NOW does it wipe out all of the last 6 days and start fresh? Or is it simply going to overwrite the oldest backup in that B2D folder?

Oh wait a second... I think I just realized how I was off with my thinking. It is the media that has the overwrite protection, NOT the B2D folder. That's where my brain was stuck on. So on day 7 the oldest media would indeed be overwritten. So a file say B2D000022 that is now 7 days old would be overwritten with new data and it would still be called B2D000022. Hopefully I'm on track with this part.

2) I currently have mine set at 25GB files. In the past I always had 1 backup to one B2D file and then I moved the file out to a new location on the post command so at any given time I knew what that file tied to. But that was too much manual interaction on my part in the long run, especially when restore time came around. I just have to get use tall all the files and not really knowing what files contain what data. I'll have to trust the system to track everything which is sometimes hard to do when your data is at risk.

3) Another example. Let's say you do a full backup that is 300GB of data and I want to be able to keep at least two full backups available at any given time. I have 850GB of storage so I setup a full backup to take place on the first Sunday of each month. I then set inc or diff for the remaining days. Would I set the full to overwrite, the ind/diff to append and then set the overwrite protection for 60 days? The first full backup stays in tact, then the second full backup would take place and then by the time the third full backup comes around it would ovewrite the first one leaving me still with 2 full backups? Would that be correct?

Thanks so much for the help.

[Greg_Saunders]

<< 1) So let's say I set exchange media to have an overwrite protection of 6 days. Then what will happen is when the job is set for overwrite it will see the protection and simply append for up to 6 days. Then on the 7th day it will actually overwrite. NOW does it wipe out all of the last 6 days and start fresh? Or is it simply going to overwrite the oldest backup in that B2D folder? >>

OR am I still off on this. Above I am saying place overwrite protection for 6 days, append is set to Infinite. And then set the job to overwrite. I was making the assumption if it couldn't overwrite it would simply append, but I think this may be incorrect according to documention... if there is no overwriteable media it will give an alert. I don't want any alerts, I want it to cycle continuously without intervention.

I would be greatly if you could clarify the overwrite protection and job settings on how things will interact.

Thanks again.

[Ken_Putnam]

I think you are still confused.

If you have an overwrite job to start things off, but then append forever, NONE of the BKF files will ever be overwritten. This is because the append just "extends" the media, so that even tho there are separate BKF files backup exec thinks of them as one big logical file.

if all your Exchange jobs are Overwrite, then when the disk fills up, backupExec will start re-using the oldest BKF files

[Greg_Saunders]

Yes, I think I am.

I think I keep trying to associate the B2D folders as a tape which I don't believe is an accurate assocation. I believe the B2D folder more aptly represents a Tape Drive and then a single B2D file (ie. B2D0000022) more accurately represents a Tape.

1) Would that be a correct association?

I am just trying to grasp how to do the following with the way Backup Exec does their protection scheme. I am re-reading the help file about overwrite / append protection. It is just very different in how it achieves it compared to the other software I had been using.

Then for me the confusion comes when thinking about what happens when there is no appendable and no overwriteable for the media called exchange when I am telling it to go to b2dexchange. In my mind I would expect it to follow what the documentation says:

<<
How overwrite protection periods are reset

Because the overwrite protection period does not begin until the job completes, the amount of time that the job takes to complete affects the amount of time until the media can be overwritten.

For example, suppose that you create a media set named Weekly with an overwrite protection period of seven days, and an append period of 0 days, and you schedule a full backup job to run each Friday at 20:00. When it is time for the full backup to run at 20:00 the following Friday, the job cannot run because the first backup job that ran the previous Friday did not complete until 21:10 p.m. The overwrite protection period for the Weekly media set still has 70 minutes remaining.
>>

I would guess from the above help file excerpt that my job would fail or not run, but it in fact it just creates a new B2D file and keeps going. That is part of my confusion on how this works. I have been trying to avoid what the documentation says, but in reality it appears the documentation is incorrect. Maybe I am reading the documentation wrong, but I had been trying to avoid that situation which made it even more difficult for me to figure out how to do what I wanted to do.

It would seem to me I could make the settings so that I can keep 4 full backups of exchange at any given time along with the associated inc/diffs.

Instead of me guessing how to make that happen, maybe someone could state exactly what settings would be needed and how it would work, because I just seem to have a blind spot on how this thing works... even after reading the help on this specific area 3 times now.

I'll guess anyway... If my condition was 4 weekly full backups and daily inc or diffs then I would do the following:

- Create media with overwrite protection on for 4 weeks and appendable for 6 days.
- Create B2D folder that will be used.
- Create Job that uses both the media and b2d folder above with full backup and set it to do an Append to media, overwrite if no appendable media is available.
- Create Job that uses both the media and b2d folder above with inc or diff backup and set it to do an Append to media, overwrite if no appendable media is available.

2) Do I have it correct now??? What I would expect it to do now is create a full backup on week 1, create inc or diff during the week and appending it to the first B2D file, then when week 2 hit it could no longer append nor overwrite so it would create new B2D file (full backup) and then the cycle starts again until the 4 weeks passed on the first (full backup / weekly inc-diff).

I guess another option would be to have 0 append like this:

- Create media with overwrite protection on for 4 weeks and appendable for 0 days.
- Create B2D folder that will be used.
- Create Job that uses both the media and b2d folder above with full backup and set it to do an Append to media, overwrite if no appendable media is available.
- Create Job that uses both the media and b2d folder above with inc or diff backup and set it to do an Append to media, overwrite if no appendable media is available.

I guess the difference here would be every single backup including full and inc/diff would create its own B2D file and be protected for 4 weeks. This may be simpler, because as i was testing the 6 days appendable I found the 6 days could start in the middle of the week if that is when I create the job and run it... then the all appendables for the first week would not be in the full B2D of the week due to when the job first started from the get go. Arghhh.Message was edited by:
Greg Saunders

[Ken_Putnam]

think I keep trying to associate the B2D folders as a tape which I don't believe is an accurate assocation. I believe the B2D folder more aptly represents a Tape Drive and then a single B2D file (ie. B2D0000022) more accurately represents a Tape.

Closer - each set of associated BKF files can be though of as a tape volume. That is if you have defined max file size as 1 GB, and the job creates four, then another (append) job adds four more, all eight are governed by the same APP and OPP times. they are one FAMILY is I think the way that Veritas puts it.

In the quote from the help file. yes, I always use one day less than you would think. OPP on weekly jobs is 6 days vice 7, monthlies 27 days vice 28, etc

if you are pointing your weekly exchange jobs to a folder called EXCH-WEEKLY, and the max disk space (not file size) for the folder has been reached, and none of the Families is past the OPP time, the job should pause on a "Insert overwritable media" alert.


I think your proposal would would EXCEPT the OPP is figured from the time that the last APPEND job ends, so you would not want a 4 week overwrite, but a 22 day OPP (prior job ends on Sunday morning, three weeks later on Mionday the Overwrite job starts

I'd set the Full to Overwrite only and the DIFF/INCR to Append, Else Overwrite. You could use the same B2D folder, since the intermediate files depend in the fulls.


Is this getting any clearer?

[Greg_Saunders]

I think I am almost there.

So would you recommend USING an Appendable period or just leaving it at 0. If I leave it at 0 then I know each will simply be overwritten after the OPP expires.

The problem with setting one... for example for 6 days to handle the inc/diffs is if I create and then start the job the first time on say a Wed, but Sunday is when the full backup takes place then technically everything will be sort of out of sync. Meaning if my goal was to keep a single FULL with it's associated DAILY's... and I didn't initially start this until wed or thurs then what you will end up with is the following:

- Wed: Inc/Diff starts (since full doesn't happen until Sunday) and this will create the first B2D file (BKF). It will have an append period of 6 days and opp of 22 days.

- Thu: Inc/Diff again and it will append and we will have 5 days append left and still 22 days of OPP

- Fri: Inc/Diff again and it will append and we will have 4 days append left and still 22 days of OPP

- Sat: Inc/Diff again and it will append and we will have 3 days append left and still 22 days of OPP

- Sun: This is when I would normally want it to NOT append, but to create a new BKF so as to keep the appendable with the full backup, BUT this full will simply append to what happened Sat. and we will be left with 2 days append and still 22 days of OPP.

OR should I set the Full to always be overwrite while the inc/diff are append? Would setting the full to overwrite always force a new BKF? And if it does where would the next inc/diff append go? Would it append to the one with 3 days append period (Sat.) or would it append to the newly created Full. Sorry for such confusion.

So to me the only way to keep the inc/diff's with the full is to make sure we don't do a backup until the first full day, which isn't really practical since I need backups to start the very first day the data becomes available to the users... UNLESS what I mentioned about forcing the full to be overwrite solves things.

??????

In my mind I would almost rather just set the appendable to 0 so that I simply have a B2D folder full of BKF's that represent 1 BKF per Backup (full or inc/diff).

I don't know why this has been such a struggle in my head.

[Bradley_Willads]

Greg,

I believe you're almost there. If you don't set an Append period, and you enable Preallocation (check the box to allocate the space for each b2d file), you will end up with a lot of wasted space. That is because the B2d file will take up 4GB (if you have 4GB defined for your B2D file size), and you may only use 2GB of it. If you want to do this, I recommend not turning on that option on your B2D folder.

If you set your Full to overwrite, and your Inc's & Diffs to append, that would be the ideal solution. The full will always look for a B2D file to overwrite.

Does that make things clearer?

[Greg_Saunders]

Just following up to see if I can get confirmation on my last post above... also just wanted to add that it seems all backups are taking place so I feel we are protected, just need to make sure all of this overwrite protection settings are now correct so we can have as little intervention as possible. Then I'll delve into CDP.

[Bradley_Willads]

Greg,

I'm not sure what question(s) you are referring to in your past posts. Can you please repost that question, such that I'm answering the correct one? Thanks!

[Greg_Saunders]

I actually missed your last post on this thread, but it didn't clear it up. I am not restricting the size of the B2D at this time. I am hoping with overwrite protection and append periods that it will become automated and not fill up the drive, unless the data grows a large amount which would require adjusting the opp and append period.

Anyway... below is the last post in this thread that I wanted to get some further comments on. You had stopped replying earlier and then Ken picked up on things so I don't know if you have seen the other posts leading up to this.

Ok... take a look at the post included here... I decided to alter it quite a bit to hopefully be more clear.

*****************************
In this example let's assume we want to do a full backup on Sundays and an inc/diff on Monday through Saturday. Let's also assume this example has 22 days for OPP and 6 days append period. Why did I pick the OPP of 22 and append of 6? My goal is to have 4 weeks of full backups along with the inc/diffs in a B2D folder at any given time.

1) Is it reasonable to want the full backup and the inc/diffs for the week to all be contained in a single BKF or (family) of BKF's??? Is that a goal we should even strive for?

2) I ask question 1 because in the B2D type media world one could ask why append at all? What does it matter whether you append to it (especially if a single full backup could span 10 or 20 BKF files due to setting the BKF to say 25GB each). If you looked at the B2D folder you couldn't tell by eye where something was an append or a full so why would it matter? Part of this question stems from being so use to the tape world where we tried to keep a full backup and all of the inc/diffs on one tape and then the next full backup started with a new tape. It just made more logical sense so you knew exactly what tape had what... even if they had been pulled offline and stored somewhere. When the backups spanned tapes and you went to take them offline and store them somewhere it was more difficult to know what the tapes had on them without paying close attention to things (especially when multiple techs were responsible for the backups)

Let's stop here and deal with these 2 and I'll drill down further.

[Bradley_Willads]

Greg,

1.) You can do this. If you don't do this, the problem you may run into though is that you will find lots of smaller files, which in turn can really fragment your drive. This can slow backup speeds down, if you don't defragment you drive regularly.

2.) If you make your B2D file sizes comparable to tape sizes, you could achieve the same behavior. I wouldn't recommend very large file sizes though, only because SMB can have trouble reading them. The best way to look at B2D is that each file is a tape, albeit a smaller tape. If you have your media set properties set up properly though, you shouldn't ever have to worry about how many files are created, and what sizes they are. As media becomes overwriteable, it will be overwritten by the Full overwrite job. By default BE wil always overwrite the oldest piece of media.

Getting any more clearer? =) I know it's a very confusing topic...

[Greg_Saunders]

Yes... minute by minute it is starting to unfold.

Ok so let's assume appending the inc/diffs to the full backup is beneficial and recommended.

Let's still assume the following: We want to do a full backup on Sundays and an inc/diff on Monday through Saturday. Let's also assume this example has 22 days for OPP and 6 days append period. Why did I pick the OPP of 22 and append of 6? My goal is to have 4 weeks of full backups along with the inc/diffs in a B2D folder at any given time.

Let me provide a chain of events of what I think would happen and let me know if I am getting this correct.

I create a new job on Wed where I have set Full Backup to take place on Sunday and inc/diff on weekdays. The OPP is 22 days and the append period is 6 days. The first backup takes place Wed since I just created it and need backups to start asap.

- Wed: First backup takes place and creates B2D000001 (assuming this one file is allowed to be large enough to not break into smaller ones for this example). This now has 6 days of append and 22 days of opp.

- Thu: Inc/Diff runs and appends to Wed. backup B2D000001. There is now 5 days of append and still 22 days of opp.

- Fri: Inc/Diff runs and appends to backup file B2D000001. There is now 4 days of append and still 22 days of opp.

- Sat: Inc/Diff runs and appends to backup file B2D000001. There is now 3 days of append and still 22 days of opp.

- Sun: Now this is when the full backup should take place and I would NOT want it to append, but to create a new B2D file so as to keep the inc/diff data with the full backup.

1) Would this full backup simply append to the B2D000001 file since there are 3 days of append left???

2) OR should I set the Full backup job to overwrite??? Would the overwrite setting automatically create a new B2D file (assuming OPP had not expired on the last one)???

3) If I should set the full backup to overwrite and if it does start a new B2D file (B2D000002) then I still have 3 days of append left on B2D000001 and now I would have 6 days of append on B2D000002. So on the next Monday when the inc/diff runs would it append to B2D000001 or B2D000002 since they both have appendable days left?

Hopefully you see what I am getting at.

Thanks

[Greg_Saunders]

I think I have just answered this doing a small test.

<<
1) Would this full backup simply append to the B2D000001 file since there are 3 days of append left???
<<

YES if the job was set with append, NO if job was set with overwrite.

<<
2) OR should I set the Full backup job to overwrite??? Would the overwrite setting automatically create a new B2D file (assuming OPP had not expired on the last one)???
>>

I believe I should set the Full Job to overwrite and it indeed created a new B2D file (assuming OPP had not expired)

<<
3) If I should set the full backup to overwrite and if it does start a new B2D file (B2D000002) then I still have 3 days of append left on B2D000001 and now I would have 6 days of append on B2D000002. So on the next Monday when the inc/diff runs would it append to B2D000001 or B2D000002 since they both have appendable days left?
>>

Well it looks like the append would have indeed shown up on B2D000001 and NOT B2D000002 which is where I would have naturally wanted it to tie to since it was an inc/diff for the full backup that created B2D000002.

Here is what I did with my small test: I created a test media with 1 day OPP and 1 hour Append.

- The first backup created a new B2D file B2D000058. This simulates Wed. backup from my original example.

- My first append went into B2D000058. (This simulates Thurs through Sat inc backup in my example)

- Then I did another full backup (with overwrite) and it created B2D000059. At this point both B2D media had OPP of 1 day and append period of almost 1 hour. (This simulates my Sunday full backup)

- I then I did another append which went into the first B2D file B2D000058. (This simulates the next Monday inc.).

SO... my point is if you setup the OPP and append for the purposes of trying to keep your full backup and associated inc/diffs together in their own B2D File family then you will have problems if you start it on any other day/time than the full backup day. If I had started my example with the full backup on the Sunday start time then things would probably have stayed in sync, but since I created the job on a Wed. (in this example) the OPP and append did not suit with keeping the full and inc together.

Hopefully you see what I am getting at. That is why I had asked should it be a goal to try and keep the full and inc/diffs together with the same BKF family, because you could easily get the inc/diff of one job to show up on a previous Full Backup BKF file.

Am I confusing you at all now. :)

[Bradley_Willads]

Greg,

Your points are all correct. However, there is no way to tell BE to go to a specific B2D file as well as a specific tape. It will check to see what's available, and use that media.

If you did start on Sunday, you are correct, you probably wouldn't be in this pickle and everything would work itself out properly.

Disk is a little more tricky, because you can't pull disks out of you server, not as easily as tapes.

One other side note, OPP actually starts at the end of every backup job. If the tape or file is loaded and appended to the next day, OPP resets itself. For example, using your 6 day Append period, 22 day overwrite in this example

1. If you Overwrite the media on Monday, the append period is now set to 6 days, OPP set to 22 days.
2. The next day goes by, and you don't run backups. Append period is now 5 days, and OPP is 21 days.
3. The next day goes by, and you don't run backups again. Append period is now 4 days, and OPP is 20 days.
4. The next day goes by, and you run a backup that appends to the media. Append period is now 3 days, but OPP resets itslef to 22 days.

Does that help clarify things more?

[Greg_Saunders]

I think I fully understand it all now... the little tests I did helped a lot as well as yourself and Ken's help.

So the moral of the story for me is if you really want to keep your append jobs tied to your full job then you need to make sure that you begin your backup on the day you want it to start the OPP/Append periods so that they stay in sync. Otherwise you may find your append jobs (inc/diff) stuck in last weeks full backup instead of the latest full backup.

I realize for the purposes of Backup Exec it really doesn't matter since it knows where it is and can track it... which leads to the question...

I wonder how many people try to set the append period for the purposes of actually trying to keep the appends with the full B2D???

What are other people doing for OPP and Append with your typical weekly or monthly full backup and inc/diff's in between??? I would love to hear what others are doing.

I think setting append to 0 would make things more simplistic and you always know you will get a new BKF file for a backup, but as you mentioned if these files are small it could lead to fragmentation of the drive, but I defrag regularly. I'm going to try and sync the OPP / append on the start date of the jobs and see how it goes.

Again I would love to hear how others are doing it!!! Or what Symantec suggests as a common practice.

Thanks

[Bradley_Willads]

Greg,

Symantec doesn't really have a common practice, only because different customers are required to keep there data for different amounts of time. Some people need to keep years, while others only need a week or two. it really depends on what your needs are.

I'm glad to see you understand how it works now. It is a bit of a tricky topic, but once you do understand it, it does make sense. :)