Sizing the Backup Exec 2010 Deduplication Database

[JimmyTheGeek]

I've been researching all of the documentation I can get my hands on about the Backup Exec 2010 Deduplication Option, and poring through the Symantec Open Community, but have not yet found the answer to the following question:

What size should the deduplication database be sized for?  Put another way, for every terabyte of data created in the deduplication folder, how much space will be consumed by the database folder, providing that folder was directed to a different disk that the deduplication folder during creation?

I notice that there is a Backup Exec 2010 Deduplication White Paper, but I get a browser error when I try to download it.

Any help or insight would be greatly appreciated!

[pkh]

You can only define 1 dedup folder on each disk volume.  The recommendation that I got from Tech Support is to dedicate the whole volume to the dedup folder, i.e., define it to the maximun size.

[teiva-boy]

The dedupe database grows at a rate of up to 10% per TB.  It's actually 6-7%(officially), but I've rounded it up to 10% for cushion.

[JimmyTheGeek]

(tevia-boy) Thanks for your response.  10% seems very high considering the testing I have done.  Even 6-7% (officially) seems high.  Can you point me to the official recommendation?  I can't find it anywhere.  I am getting ready to release a Backup Exec Deduplicating Appliance and need to configure the database size appropriately.

BTW (pkh), to respond to your post, not only can you only have one dedupe folder per volume, you can only have one dedupe folder per media server.  I agree that dedicating said volume to the deduplication effort is best practice.  However my post relates to the dedupe folder database, which is recommended to be placed on a separate volume.  I am trying to determine the sizing algorithm for that structure so that I can allocate enough space for it.

[teiva-boy]

This is based on official training given to resellers by Symantec over the last year.  It was just something I noted down in my notes.  

The catalog is going to hold a unique MD5 hash for each segment.  One file is made up of multiple segments.  Thus the database will have many more records than I  can possibly count, and much more than the SQL Express instance will ever have.  16TB max dedupe store, close to 1TB of dedupe database.  Then again, I question if this has ever been tested in BE if ever?  The technology came from Symantec PureDisk running on Linux, now it's ported to Windows and running on BE.

Also with R2, some of the hardware requirements changed like for example, 1.5GB of RAM per TB of deduped data BE is managing.  So a fully populated server, holding 16TB of deduped data, would have at least 24GB of RAM, if not closer to 32GB for OS overhead, services, etc.

On the bright side, this could be Symantec's way of being overly cautious and the Db will be more like 2-4%...  I'm sure it was sized for the worst possible scenario.

[deejerydoo · Accepted solution]

Hi Jimmy,

Just stumbled across this post, which is similar to a question I was asking about more advanced info on the dedupe option. Another forum member (ItsMeAffinity) attached the deduplication option whitepaper I think you were having difficulty getting hold of.

Take a look at my thread if you still need to get hold of this.

https://www-secure.symantec.com/connect/forums/more-advanced-information-regarding-dedupe-backup-exe...

Regards,

Deejerydoo

[itsmeaffinity]

Hi

That is right it is one of the white paper talks about the FAQ on dedup

If this helps you please mark this solved.

Keep posted ;)

Thank You

[JimmyTheGeek]

Thanks for the white paper link, it confirms another source of information I had found. I've marked this question as solved, and I appreciate your response.

regards,

JimmyTheGeek