Backup Exec 2010R2- DeDuplication space

[bjhow]

I am currently using the De-Dupe option on BE 2010R2- which is working pretty slick. I have a 8TB NAS device that I am using for a dedicated volume. It' been running since Nov, 2010 with pretty much no issues, except for when I reboot the Backup Exec Media server, the DeDuplication Folder shows offline. I have to stop and restart the services again to get it to come online. Other than that, it seems to be working fine. The only concern is I'm down to about 1.9TB left out of 5.5TB. I have the data retention set for 4 weeks. It appears that the drive space has been steadily decreasing over time, which it should since I added some backups to it, but I seem not be gaining space back at any fast rate. Since the data retention is set for 1 month, shouldn't I be seeing more space as a result or am I wrong. I ran the manual reclaim process that Symantec offers in a KB article, but when I get to the crcontrol--compactstart, I get no response on the screen. It comes back with blinking cursor and nothing appears to be happening at that point, so I am not sure if it actually is compacting the database. If I look in the data folder in BackupExecDeDuplication Storage Folder, I can still see .bin files that date back to when I first implemented Dedup - 112410. All these files are 262,144KB in size. Do I have to clean these files out manually to see more space? Will that screw up the Database that DeDup uses to maintain the backups? Just wondering how to clean up and get some of my disk space back.  Any suggestions would be helpful. Thanks.

[teiva-boy]

Do NOT ever just delete files from the directory.  Bad things will happen.

The KB article is correct.  You just have to keep running the commands to have it take action, and it will take some time.

Since you are using a NAS, and I assume an iSCSI connection since CIFS is NOT supported...  My guess is that the BE services are starting before the iSCSI services are and are mounted.  Change the BE dependancies to start after the iSCSI service started.  There are numerous how-to's on the web for this, and very common for MS appliances like Exchange and SQL.

[bjhow]

OK, I thought just deleting the files would be bad. It appears DeDupe uses PostGres SQL and so I didn't want to mess with that. I will keep running the manual reclaim process and see if that frees up some space.

It is an Isci NAS and I have it setup in Isci Initiator to mount that volume/device when Iscsi intiator starts but still glory there. I just stop and start the services again for now and it brings the De-Dupe folder back online. I check to see if there is anything on BE dependencies as well. Thanks.

[macpiano]

BE won't allow you to delete those files no way no how unless you de-install BE itself or the dedupe option. Goodness knows I tried. It always says they are still in use. Even if you delete the dedupe folder from BE.

I have found that if I click on one of the servers that uses the dedupe fodler and just look at the properties that the dedupe folder comes right back online. I think this is a bug.

[deejerydoo]

Once disk space has been allocated on a disk for deduplication storage (container) use it cannot be freed up again. There's a tech note from Symantec about this here:

http://www.symantec.com/business/support/index?page=content&id=TECH154127

Even if you follow this KB article to reclaim/compact space, the space is only freed up withing the containers in the dedupe store. No space is returned to Windows:

http://www.symantec.com/business/support/index?page=content&id=TECH130103

This also has a negative effect on any copy set backup jobs you might do for DR of your deduplication store (as is Symantec's best practice) as it is very easy to exceed your off-line backup media's capacity.

If you ask me this is a major limitation in the deduplication systems usability. It's all too easy to overlook this limitation and once you're through your storage ceiling there is effectively no way out, other than to completely remove the deduplication option, reinstall it and start again.

This has GOT TO BE A VERY HIGH PRIORITY for Symantec to improve the manageability of their deduplication store in future releases.

[RpR]

What a major bummer, I'm in the same situation as the OP here, and I'll be running out of diskspace soon if it keeps growing at the current rate.

Is there any other practical way to move the data from the dedupe folder, with history, to for instance tape, and then recreate it, but without the wasted space? I'm wasting over 2TB's here!

[teiva-boy]

Stop services, move all folders to new directory/volume with more space.  Assign same drive letter/path as before.  Start services.  There is a KB article that more or less outlines what I briefly covered.

Moving to tape wouldn't work in this case.

[deejerydoo]

Hi Teiva-Boy,

That will relocate the deduplication store to a larger capacity volume, but it wont reclaim any of the wasted space. RpR's question is specifically about reclaiming the otherwise wasted space that the deduplication store grabs.

As far as I know, Symantec do not have any sensible management control over the deduplication store in Backup Exec as yet.

I have run through their rediculously lengthy (over an entire weekend) process for reclaiming space within the deduplication store, on more than one occasion. While this helps my particular issue, with the off-line media capacity limitation, it still does not reduce the overall size of the deduplication store as far as Windows is concerned.

Deduplication uses it... Windows looses it! For good!

I ihave more than a sneaky feeling that 2012 will not improve this situation...

Cheers,

David