This article applies to Backup Administrators working on Backup Exec 2012, 2014 or the latest BE 15 with latest patches applied.
Every Backup Administrator struggles with this question of Storage Management at some point. Even though disks have become less expensive with time, data has also grown exponentially. So, it is always a challenge for the administrator to ensure his Disk Storage is managed and utilized optimally. Generally, disk storage is used for primary backups whereas secondary backups go to a device that can be potentially moved offsite in a storage like traditional tapes. Managing tapes is another task and out of scope of this article. As far as storage management is concerned, an administrator's biggest challenge is managing space on the disk storage where the backup sets (Restore/Recovery Points) are stored and also making sure all his data is protected and recoverable without compromising on Recovery Time.
In the previous versions of Backup Exec, we had to keep those disk media under Media Sets while understanding the concepts of Advanced Device and Media Management (ADAMM). Those days, the retention was tied to media, i.e. B2D00000x and there were concepts like Overwrite Protection Period which would determine when a media becomes available for overwrite. This was difficult to manage and Administrators had lot of logistics to handle when deciding which media goes where. For example, a backup job of 10 GB would use up 10 .bkf files or media and an administrator would have to manage them while considering the period for which they wanted to retain these media and we also made things more complex by appending to existing media ending up with situations wheres we were writing two backup sets to single media. This made the job of an administrator more difficult.
With this problem in mind, Backup Exec 2012 introduced the concept of automatic Disk Management in an attempt to make Backup Administrator's job easy.We wanted to manage Disk Storage ourselves. In an attempt to achieve this, we even made the Disk Storage abstract to end users i.e. no more displaying of Media for Disk Storage. The idea here was to make sure an administrator doesn't have to spend time understanding the media or the files we created on the disk storage. All that an administrator is concerned about is his recovery point or the backup set. Hence, we completely removed the media management part for Backup Exec from the User Interface and took things into our hand. All an administrator would see is a recovery point or backup set which of course went into a set of media. For space management, we introduced the concept of Data Retention for a backup set while doing away from the old complex concepts of Overwrite Protection Periods etc. And for obvious reasons, we did not want to do any append operation on a media. An administrator now just has to deal with the backup sets in UI and we do every thing else under the covers. This brought us to the next requirement of automatic Disk Space management. Now an administrator would set backup set retention time for a backup set by configuring it in a backup job as shown below:
Figure 1:
Once an administrator sets this to 2 weeks, we know we have to keep this set for two weeks and then let it go and such set whose time is up, is called an expired set. For the beginners a backup set is a unit for recovery point. For example, if you are backing up C:\ in a job which runs at 11 AM, we create a backup set for C:\ and mark it with 11 AM. We also know which media (B2D0000x.bkf file) this backup set used. So, an administrator gets a backup set under his disposal after the job has completed. Now, you can go to your storage tab and look for your backup set as shown below:
Figure 2:
Notice the expiration column there. That tells you when this backup set will be available for expiry and reclaim. Similar view is also available under the resource container or the server that was just backed up. Check this picture below from the backup and Restore tab:
Figure 3:
How cool is that? We now have a backup set which tells us when it is going to expire. If you want to expire it sooner, just right click on it and hit expire and we will do that for you. And if you don't want it to expire, we will let it sit there till the time comes and we are informed that this set is ready to expire. Obviously, we keep doing this periodically. This automatic expiry of backup sets is done every 1 hour when we reclaim the space used by Backup Sets that have expired. Every 1 hour, we keep running this reclaim process to make sure you don't end up using up all your space on the disk. But, you know we are also concerned about your data and would not want you to lose any set mistakenly. Hence, we have this small safety netnet of not deleting your last and only backup set for certain resource. For example, if you have run Full backup of C:\ 10 times and all those 10 are expired, we delete only 9 of them while retaining the latest set. Don't like it? Ok, you seem to be too conscious about your disk space and are fine with us deleting that set for you.. Tells us and we will delete that too, here is your check mark:
Figure 4:
Be careful when you set this though. With this checked, we will delete all sets we find expired for any resource. Once a set is expired, we change the color of that set in UI to blue so that an administrator can easily distinguish it from other sets and knows that next cycle of Data LifeCycle Management (DLM) will delete this set. Remember that every 1 hour task? That is what we refer to as DLM task. It runs every 1 hour, executes simple SQL query to find out the backup sets that are expired now and delete them from database and also clean up the catalog and data from disk. DLM is capable of deleting all types of media including the GRT backup sets, like IMG0000x folder. It also can manage the backup sets on a Dedup Storage. So, we manage everything that goes to a disk.
How about RDX storage? Yes, of course that is also a disk, we know how to manage it. There is a catch though. We don't delete Backup Sets on RDX cartridge at 1 hour frequency. It is instead done when a backup job is triggered. What if an administrator intentionally disconnected the disk and brought it back to restore data? Do we still delete that data if we find it expired? No, we have a safety net there.. Check this global setting:
Figure 5:
Here, there are two settings. One for Fixed Disk and other for an RDX Cartridge. Defaults are higher for cartridge. When you leave it to default and get a Disk cartridge back in less than 30 days, DLM will apply and we will delete expired sets. However, if it is out for more than 30 days, we understand the cartridge must have been taken offsite and has come back now. In those cases, we understand it is possibly something we got for restore and hence the device is marked as read-only and DLM does not reclaim the backup sets there. That is good. What if I am done restoring my data and want DLM to do what it does best? Go ahead and take it out of Read Only mode:
Figure 6:
That explains the power of Automatic storage management done by Backup Exec. This helps administrators manage their data, backup sets effectively and without much user intervention.
Now, let us talk about recovery point chain. A chain is where you have a combination of Full, incremental or differential backup sets. Here we consider these backup sets as a unit because from recovery point of view we will need other sets in addition to Full. When it comes to reclaiming of space we have to be careful of not deleting a set which has some dependency. For example, if a Full Backup Set has expired whereas incremental is not, should we delete the Full set? If we do, the incremental will be of no use. In such case, we wait for all sets to expire before we delete the Full. We call this kind of sets as dependent sets and it is also called as recovery chain. So, the bottom line is that we do not delete any dependent sets, unless explicitly mentioned. So, if a user choses they can tell us to expire all dependent sets when manually expiring a set. This overrides our automatic logic and we would delete all dependent sets in additoin to the one selected for expiring.
Alright, what if I want to retain some sets longer than their desired retention time? We can chose to retain the set by right clicking on Backup Set and chose to retain, which gives us option as shown below:
Figure 7:
Once, we chose to retain a backup set, DLM cycle will not delete it.
Let's go through this again and summarize key points about DLM:
1. DLM stands for Data LifeCycle Management. It is an automated way of managing your disk storage.
2. The Data Life Cycle is of a backup set and an administrator does not need to look at media at all.
3. When configuring job, decide retention period for the sets that will be generated after the job has completed. Check Figure 1 above.
4. Backup Sets can be seen from under the storage or the Resource Container (Protected server). Check Figure 2 and Figure 3 above.
5. Backup Set has "Expiration" property which tells you when the set is to be expired.
6. Backup Sets are marked as expired when the retention period ends. The color of the sets changes to blue when that happens.
7. DLM cycle runs every 1 hour and grooms all the backup sets which are found expired at that point.
8. We run a SQL query to our Database, to determine expired Backup Sets.
9. DLM manages Disk Storage (Traditional Backup to Disk Folder), Dedup/OST Storage and also Disk Storage on an RDX Device.
10. RDX reclaimed is not done automatically, it needs to be trigerred by a backup job. This is a deviation from behavior for Fixed Disk Storage.
11. We can chose to retain the sets in exceptional cases (for legal or other reasons).Check Figure 7 above.
12. You can set in the UI if you would like DLM not to reclaim stuff on a Disk Storage for cases when you are taking it offsite and may want to get it back to the server after certain time for attempting restore and would not like DLM to reclaim data. Check Figure 5 above.
13. The default behaviour is to leave the last and only backup set intact. If a backup job creates 10 Full backup sets and all of them are expired, DLM will delete only 9 of them and leave the latest of those. You can have DLM delete the last set by selecting an option in UI (Check Figure 4 above), explained above. By doing this check box, you are telling DLM to delete all expired sets without any safety check. This needs to be done with care..
14. DLM does not delete dependent sets. If a Full and incremental set is present, you cannot delete the Full without expiring and deleting the Incremental Backup sets.
15. DLM does grooming of all types of media, i.e. GRT and non-GRT.
16. DLM does not work for Tapes. Tapes still continue to use Media Set properties as with traditional versions.DLM is only to manage Disks.
17. Disk Media is hidden from users and their management is also handled by Backup Exec. An administrator should never have to look at the content of Disk Storage. Instead, you just manage Backup Sets in UI.
