how to manually reclaim dedup space

[manatee]

NBU 7.7.3

i have 9TB partition where all backup images are stored (MSDP?). now, there is only 1.2TB free space and because of that it is affecting the backups. i have to deactivate development/UAT/SIT server backups to give way to production server backups.

from my limited knowledge of dedup process, i believe i can gain more hd space if i delete some of the dedup images from non-production servers. but some questions remain:

1. if i delete/expire those dedup images, would they also expire the backups contained in the tapes?
2. i have been processing transaction queues but only gained a minimal free space. nothing dramatic. is this the way?
3. how to really delete the dedup images for non-production servers? the only TN i can find is this https://www.veritas.com/support/en_US/article.TECH124914
4. lastly, is my plan advisable? like unless the ordered disks arrive, i don't have any way to add more hd space.

[sdo]

Deleting some older copies (but only the copies on MSDP) of backup images probably won't help very much at all, because those backups will probably be very similar to recent backups, and so there will be very little gains after running queue processing, cleanup and compaction.

Personally, I think you best bet would be to look for backup jobs which consistently display low dedupe rates - and target those backup clients, disks/volumes, folders, file-sets... because if they dedupe poorly, for example every week and you have four week retention... and so if you can somehow change the client side application behaviour then you could end-up making a 4 x saving (in MSDP dedupe disk usage) on those client backup job sizes. 

[sdo]

In an attempt to answer your questions:

1) if i delete/expire those dedup images, would they also expire the backups contained in the tapes?
- Yes, if you choose to expire the image ID and not specify a copy number, then all copies of a particular backup image could be accidentally expired.

2) i have been processing transaction queues but only gained a minimal free space. nothing dramatic. is this the way?
- Yes. If you've slowly been creeping towards MSDP full, then running queue processing manually isn't really going to give you much back - unless you get lucky and you just so happen to run queue processing after a big old random data test backup just so hapenned to expire and then you purged it using queue processing - and then you'd think queue processing did the trick, but all it was... was random luck.

3) how to really delete the dedup images for non-production servers? the only TN i can find is this https://www.veritas.com/support/en_US/article.TECH124914
- Look at my post above. Deleting some older copies of backups that typically dedupe very well will most probably not bring much space back.

4) lastly, is my plan advisable? like unless the ordered disks arrive, i don't have any way to add more hd space.
- Your options are:
4a) Cease backups of some entire sets of clients AND expire all of their backups from disk - but... here's a known unkown... if a bunch of clients in SetA have similar backup data, i.e. dedupe well against a bunch of clients in SetB, then again... ceasing backups for all clients in SetA and expiring all backups (on MSDP) for all clients in SetA - still won't bring you back much MSDP space - because the bunch of clients on SetB still referenced the very same dedupe blocks.
4b) Acquire more storage.

4c) Delete / remove a lot of old superfluous data from backup clients.

4d) Renegotiate what you backup (i.e. what goes in your backup policy selection lists).

4e) Backup some non-production poor dedupe jobs directly yo tape and so avoid landing in the MSDP pool.

[Marianne]

You know you need more storage... REALLY!

[manatee]

yes. that's why hard disks are on the way. but until then, i have to reclaim space manually and the only option i have is to free up or remove dedup images of some servers.

[sdo]

Got any backup jobs which regularly dedupe poorly?   The copies of these which reside upon MSDP storage might be the best candidates for early expiry, but only if if you already have copies on tape.

This script might help you:

https://vox.veritas.com/t5/Downloads/NetBackup-a-Windows-based-script-to-identify-oldest-backup/ta-p...

...which is safe to run, as it is only a query / list / report type script...

N.B:  However, the danger elements are in:

1) the interpretation of the results - i.e. ask oneself, do I know NetBackup well enough to be able to make a sound judgement about the results of the script.

2) the fact that the script might contain bugs, which causes you to make a mistake with expiry, which results in backup data loss.

.

Anyway, AFAIK and IMO it is safe to try to run as it only lists backups and then tries to work out what the oldest images are, which are on MSDP, and which have a copy on tape... but...

Note the disclaimer in the script.

[Marianne]

Do you still have the same extremely short retentions on MSDP as in the past?

[manatee]

yes, in fact, even shorter now. expire after copy (i know its a sin but that's for another story).

[Marianne]

You are wasting time, money and energy with MSDP in the current state.

Simply backup directly to tape until you can increase your MSDP pool.

Hopefully your NBU reseller can provide assistance with dedupe calculation?

[manatee]

that has already  crossed my mind but disks are faster than tapes.

as i read again the dedupe guide, looks like the reseller has short changed us with so small MSDP partition. anyway, lates update that the disks are with customs now so only a matter of few days (fingers crossed).

[sdo]

In my experience, enterprise tape drives are faster than disk within backup clients.  Hence we would even consider D2D2T, because the entrprise disk (in the middle) should be faster than most clients.

Do you have a whole bunch of non-prod backups?  Maybe you could send those directly to tape, multi-plexed, and I think you might be pleasantly surprised.