Netbackup Storage LifeCylce Policies and DD Pool filling up

[JasonP_76]

Hi all,

Recently we implemented Deduplication (and soon will be replication to another server), using Disk Pools and Storage LifeCycle Policies.

Before we had just Basic Disks for Storage Units for Online backups to disk.

Since moving to SLP we are constantly getting error 129 (Storage unit is full). We never had this issued with a Basic Disk Setup. It just worked as it should and did what the High and Low watermarks are supposed to do. With SLP these watermarks don't work at all. The watermarks are the default of 98% and 80% (high and low respectively).

Why does it not work ? Why does it not perform in the same manner ? Why is it not consistant ?

How do I fix this issue so that the server just expires images (the oldest ones) from the online disk storage when it needs the space to perform newer ones like it did before ?

Thanks

[Marianne]

SLPs work different - the retention levels that you specify in the Backup and Duplication destinations are applied to images.

You also need to confirm that duplications are actually going through - until duplications have completed successfully, the backup copy is assigned Infinity retention. After successful duplication is actual retention applied.

Have you read up about Storage Lifecycle Policies in Admin Guide I yet?

There is also this Best Practice Guide: http://symantec.com/docs/HOWTO73205

[Jaykullar]

Which deduplication pool / appliance are you using?

[JasonP_76]

I am not using an appliance. It is a PureDisk Pool created in NetBackup 7.5

[Nicolai]

You will need to configure the SLP to perform staging operations.

Go create a new SLP:

Add a Operation -> "Used for Backup". Set "Staged Capacity Manged Desired cache Period" to 2-3 days

Add a new Operation: This time for "Duplication". Retention type must be fixed this time.

The SLP best practices is really good - I recommend it highly : 

http://www.symantec.com/docs/TECH75047

[JasonP_76]

Marianne,

So what are the high and low watermarks then supposed to do then on the Disk Pool if SLP's are involved ? Do they even do anything then ?

Being forced down the route to use SLP's just to get duplication to work, and as a result completely change the way backups are retained is crazy. 

So basically you are telling me that if I back anything up using SLP's the data has an infinite retention until it is duplicated somewhere else and even if it is it holds onto its retention period and does not expire sooner even though there is another copy somewhere else ? (hence the reason why the disk is filling up, because it won't expire duplicated copies prior to its actual retention end date)

The way BasicDisk worked was so much easier and less hassle. Just set the policy, point it to a Storage Unit and leave it and Netbackup will just do everything in the background and you never have to worry about it erroring for disk space, because it just worked and complied to the high watermarks.

I have not changed anything the way we back things up from when we had it in a BasicDisk setup (retentions and staging to tape). And even SLP's seem a little archaic that you can't set when you want the Duplication to Tapes (Or any duplication, even cross site) to run like they do when you set a staging with a BasicDisk Storage Unit. They just run when ever they feel like it. There seems to be no flexibility using SLP's, and it seems your hands are tied with using them when you want to set up duplication.

[Nicolai]

Don't use SLP unless there is a gain :)

The DSSU is working really good.

SLP does provide flexibility - lots of it - But not all backup configuration need the flexibility of SLP.

[JasonP_76]

Nicolai,

I don't see any option for "Staged Capacity Manged Desired cache Period"

[Nicolai]

GUI -> Storage -> Storage Lifecycle Policies -> Right click -> New Storage Lifecycle Policy ->  Add (here you will find it)

[JasonP_76]

Nicolai,

I have to use SLP's to have Duplication of data to another Disk Pool on another site. There does not seem to any other option when I started going through the Netbackup Docs for DeDuplication and Cross Site duplication.

[JasonP_76]

I know how to get to add an SLP.

I am saying that option you are talking about is not listed in ANY of the option within creating an SLP

[Nicolai]

Is that pool within the same Netbackup domain ?. if not you need to look at Netbackup AIR

[JasonP_76]

The pool will not be in the same domain as because the source media server is also the master server and if the master server goes offline the media server it would replicate to within the same domain would not work without the master, so we have to set it up as two seperate netbackup domain's.

But that is later. I am trying to sort out getting the disk storage from not filling up and Netbackup to manage the disk usage and expire backups as and when is needed to release space for backups to work.

[Nicolai]

The only approved way to replicate data from one Netbackup domain to another is via Netbackup AIR. The use of MSDP is required.

[JasonP_76]

Nicolai,

I understand that. And we will have that in place. We already have a DeDuplication Disk Pool in place and working. Just need help to sort out the disks filling up when using this DP and Netbackup managing the disk usage automatically.

Can you help me get that sorted first ?

[Jaykullar]

NetBackup will not "manage disk space" unless you configure your SLPs to do this. As Nicoli previously mentioned, when configuring your SLP check the "staged capacity managed" and choose your desired cache period, or expire after duplication.

The other options are to either reduce your retention period or increase your disk space.

[Nicolai]

I think I can - But I am missing the bigger picture of duplication, replication and stagning. 

Can you please explain what you want to do - Is the problem the deduplication pools fill up?

[JasonP_76]

Jaykullar,

As I mentioned when configuring my SLP's there is NO option to check anything for "Staged Capacity Managed" or anything that sounds remotely similar. There is NO ability to choose any cache period. 

[RLeon · Accepted solution]

Let's start again...

From the Nbu Admin guide, SIS storages like PureDisk and, by extension, MSDP, do not support "Capacity Managed" mode in SLP:

Capacity managed is selectable for any disk storage unit that is allowed in a storage lifecycle policy. However, for the disk types that support single-instance storage (SIS), Capacity managed functions to various degrees. In order for Capacity managed to operate, NetBackup must know how much space a backup image uses. With SIS enabled on the storage unit, NetBackup cannot know exactly how much space a particular backup image occupies.
The following storage unit configurations use SIS:
 - PureDisk storage units
 - Some OpenStorage storage units, depending on the characteristics of the OpenStorage partner.

Some people only use the term "SIS" to refer to some variant of this deduplication technology that works at the file level, but it will have to be a discussion for another time...

The Admin guide is just saying - in a long winded way - that the "Capacity Managed" mode will only work on non-deduplicated AdvancedDisk Disk Pools.
BasicDisk is out of the picture because SLPs do not work with them.

When the high water mark setting of a PureDisk disk pool is reached, backups will fail, and Image Cleanup will be triggered to expire images until it falls below the high water mark, i.e., it will not necessarily reach the low water mark. Dedup disk pools do not use the low water mark setting, only capacity managed AdvancedDisk storages do, as the Nbu Dedup guide puts it:

The Low water mark has no affect [sic] on the PureDiskVolume.

You would expect that whenever the Image Cleanup is triggered by the high water mark, images will actually be expired to clear some space, but this is not always the case.
Specifically, images that have not been duplicated to the next storage down the SLP hierarchy will not be expired (regardless of its fixed retention setting), and images that have been duplicated but have not lived through its fixed retention setting will also not be expired.

To solve your problem, in your SLP you can either shorten the retention for the images that will be staged in the PureDisk pool (don't worry, they won't expire until they are duplicated to tape), or you can set the retention as Expire After Copy, which does exactly what it says.
The duplication target/destination retention (tape, in your case) can be set to a longer fixed retention. It does not have to have the same retention as the staging PureDisk storage images.

That is one of the main differences between BasicDisk Staging and using SLP to duplicate from disk to tape. In BasicDisk Staging, images in the staging disk storage and in the target storage have the same retention, as in:

BasicDisk Storage(1 month retention) ------> Tape(1 month retention)

But with SLP, you can have a different retention at every stage:

Dedup Storage(1 week retention) ------> Tape(1 month retention)

...Although with SIS/Deduplication you do lose the ability to have the (staging) disk storage running in capacity managed mode.

[Jaykullar]

Good post Rleon.