BUE 2014 Disk-to-disk & Deduplication planning

[BillClark22]

Looking for some real-world insight or pointers into a new backup system we are planning and want to test soon. 

We have recently purchased Backup Exec 2014 with V-Ray licensing, Windows agents and several Application & Databases agents.  We are updating our current Backup Exec 2012 system and adding in disk-to-disk and deduplication to the plan.  We purchased a 48TB external RAID unit that will connect directly to our BUE2014 server via a 6GB SAS link.  Also directly attached to the BUE2014 server is an LTO-5 library, also external SAS.  Will be backing up VMware virtual machines, physical Windows servers w/ SQL and Exchange(virtual).  All in all, about 1.5TB nightly, and 3TB for a weekly "full" job.

Been reading about deduplication and disk-to-disk and the Symantec Best Practices guides recommend different disk storage for both, so I'm thinking of splitting my external RAID unit 60/40 (D2D/Deduplication).  I'm thinking of backing up everything to disk using multiple jobs, then other jobs to go from the disk backup to tape for offsite storage.  When does the deduplication come into play? How does deduplication work if I want to do whole virtual machine backups?  Thanks.

Bill

[pkh]

Dedup is another form of disk storage, in which only unique data chunks are saved, thus saving disk space.  For dedup, a file is "broken" up into data chunks and each chunk is compared to chunks which are previously stored in the dedup folder.  If the chunk matches an existing chunk, it is not stored.

For normal disk storage, no such comparision takes place and everything is stored as it is.

For your backups to disk, you can use either one.  Of course, dedup uses more resources, like CPU and RAM.  This is the trade-off for your disk space savings.

[BillClark22]

I understand how dedup works in theory for standard files, data, etc.  But how does it come into play for backing up an entire VMware virtual machine?  The virtual machine is only made up of a handful of files as opposed to the thousands in a physical install.  I know you can actually backup data files, etc of a virtual machine just like a physical one, but I'm more interested in doing full virtual machine backups of key VM's that we have.  Does BUE2014 with V-Ray and the dedupe option go "into" the virtual machine to see the files just as a physical machine?  If not, I don't see how the dedupe option would benefit or even work in regards to whole virtual machines.

In addition, I'm planning on starting backup jobs that will backup from servers to the disk array overnight, then during the business hours run copied jobs to take that data to tape.  Trying to figure out where to slip dedupe into this.  Any suggestions here?

Bill

[pkh · Accepted solution]

BE does not go into the VM, thus the dedup ratio for VM's would be poor, but you can still save to the dedup folder. For what you describe, you either backup to a dedup folder or to disk storage, there is no "slipping in" of the dedup folder

[BillClark22]

Ok, so I've split my array into two volumes, one for D2D(24TB) and one for DeDupe(11TB).  So basically I can do full machine backups of my virtuals directly to the D2D volume and for my Windows-style backups of data, files, etc. I could send those to the dedupe volume that is setup as a Deduplication Disk Storage within Backup Exec and utilize that feature to try and reduce the size.  And after both types of jobs, I can run a linked job to then backup both to tape?

[pkh]

If both backups are in the same job, then you can have a duplicate stage to duplicate the backups to tape.  If not, then the only way to do it in one go is to do manual duplication.

[teiva-boy]

So much missing and/or bad info in this thread in regards to deduplication.

BackupExec and other Symantec backup products have used Application streamer handlers for years now. The stream handler is Symantec's way of knowing what a particular data type is.  e.g. VHD vs VMDK vs Oracle, vs NDMP-NetApp, etc...

Without these stream handlers, all data is the same to Symantec, and dedupe ratios will be pretty poor.  See BackupExec 2010 and PureDisk as great examples of that.  With each service pack or revision, Symantec has gotten better at stream handlers, and understanding a data format.  There is still a HUGE disparity between NetBackup and BackupExec though.  The good thing is that with BE2014, stream handlers for VMDK's and VHD's are now available.  yay, it only took 4 years.

That means BackupExec 2014 has the ability to deduplicate VMware and HyperV files better than before.  Maybe actual dedupe and not just better than compression rates.  It can see the data type is a VMDK and dedupe across different VMDK files.  

So at the end of the day, you should be putting all data into the dedupe pool first.  Then back out and go to plain disk if needed for other reasons (slow restores, GRT not functioning properly, etc)  There are MANY occasions where you may just write Exchange backups with GRT enabled to disk over dedupe.  Or write to disk for just a few days worth of retention, but clone to a dedupe pool for slightly longer retention since it should be more efficient, space-wise.

I've been dabbling with their dedupe for years as an employee and now elsewhere, and the dedupe works, when there is a stream handler.  It's terrible when it doesn't have one.  So terrible, that I just refer customers to use an OST appliance like DataDomain or Exagrid instead most days when a customer talks Symantec + Dedupe.  Then the stream handler excuse is eliminated for all data types.  OST appliances just do it better...  Not necessarily cheaper, just better.