Dedupe performance, What should I be getting?

[smtp25]

We are doing VM agent backups of vSphere 4.1 virtual machines (hardware v7) and are not very impressed with dedupe backup performance.

What should we be getting??

Backups running: CPU's are mostly idle, memory has plenty of GB's free, queue length for DedupeFLDR near 0

Backup Exec 2010 R2 SP1 and all available updates from LiveUpdate
DL385 G7 dual-12 core cpu's (total of 24cores)
32GB RAM
Windows 2008 R2 (all updates)

Transport is: SAN

Dedupe Folder Storage: 11 x 1TB drives in a RAID6 (IBM DS3512 - dedicated to one media server)
Dedupe DB: 11 x 2TB disks (N3400 [i think] - also used for hosting vmware)

Dedupe performance - varies from 1500 to 2600mb/min.  Typically though the max with more than 1 VM backup running is 2000mb/min total across all jobs.
Backup-to-disk performance on same disk 3000mb/min
 

There other site with same server config and IBM XIV for DB, Fldr and hosting VM's and gets the same performance.

Any one able to share there figures?

[CraigV]

Hi,

Don't confuse B2D performance and dedupe performance. They're like chalk and cheese...

Dedupe is going to look at the file/block level and backup accordingly, and is more CPU/RAM intensive as a result. B2D doesn't care about that...it just backs up the files. This is why you have a difference in the speeds.

That said, I take it those 2TB drives are SATA? SATA doesn't run at the same RPM as fibre-channel/SATA drives, so again your performance won't be indicative of how fast it can run.

It depends on how you have set up your drives as well...RAID6 would be slower to write than RAID5 for instance due to the extra parity bit that needs to be written. I believe that a lot of the performance you can get out of dedupe depends on how you have configured your drives, strip size etc.

I've put the BE 2010 Dedupe Best Practice Guide in below...give it a read, and then post back here with any further comments/queries...

http://www.symantec.com/business/support/index?page=content&id=HOWTO21767

[teiva-boy]

If anything, when performing vmware dedupe, you should see on the media server at least one core being used and not to exceed 85% usage.  

The dedupe engine is not multi-threaded and cannot take advantage of multiple CPU's or cores.  You need to create multiple-concurrent jobs to maximize performance.

That said, you'll never approach the performance limits of your media server with Backup Exec.

[smtp25]

Appreciate the responses guys - What throughput speeds are you seeing for multiple concurrent dedupe vm agent backups?

@CraigV:

"Don't confuse B2D performance and dedupe performance. They're like chalk and cheese."

The B2D was to the same disks as the Dedupe Folder, so shows that the disk subsystem is capable of more than I'm seeing with a dedupe backup.    

"Dedupe is going to look at the file/block level and backup accordingly, and is more CPU/RAM intensive as a result. B2D doesn't care about that...it just backs up the files. This is why you have a difference in the speeds."

Agreed it is more cpu/memory intensive but barely seem to be used.  

"RAID6 would be slower to write than RAID5 for instance due to the extra parity bit that needs to be written. I believe that a lot of the performance you can get out of dedupe depends on how you have configured your drives, strip size etc."

Queue length during dedupe is near 0.  I have run IOMeter with 64k seq write and get about 900gb/hr (If i recall correctly).  disk formatted with 64k blocks, is sector align (as its Win2k8)

@teiva-boy

None of the cores get to 80% and I'm running 6 dedupe full vmware agent backup jobs concurrently.  I could reduce that number and see the speed difference but given one concurrent job only never exceeds 2600mb/min and degrades if another is started.

[teiva-boy]

You have a bottleneck somewhere on your disk storage.  2600MB/min is about 40 something MB/s, which is slow.  I can get that with a single desktop SATA hard drive.

Either your volumes/LUNS are configured inefficiently, or perhaps you have a network bottleneck? from the N3400.  Keep in mine that IBM NAS is just rebranded NetApp, so look to NetApp for performance guidelines.

I've gotten anywhere from 4000-6000MB/min on an HP EVA4400, which is about 100MB/min.  Even then, it could go faster if I had more spindles...