Slow Backups on deduplication storage

Hi,

I edited this post...check and make sure that any AV running on that media server isn't perhaps scanning the dedupe folder, and if it is, exclude it.

Thanks!

There is an antivirus running on server (ESET NOD32)  but deduplication folder is excluded from File Scanner (Symantec Backup Exec installation folder is too)

Any other idea there?

Job rate is really awful. It can take days just to backup a single server :(

What speed are your NICs and switch ports set too?

1Gb both

Already tried to fix speed or set port to auto, same result

I am actually working on this with Symantec support. He told me that split IO/Sec value seems to be really high on my system between 100 and 600 most of times

Seems to be related to number of disk or stripe size (256K actually on disk storage). He wants me to do additionnal tests copying a lot of small files from network, but I already know what would be the result. It would be really fast compared to backup exec job rate :(

I met the same issue with my Dell R510 server.

32GB memory, 2 processors and array twelve 7.5k hard disks in a raid 5. 1GB network

Jobs rates now are most around 400 Mb/Minute which is really poor.

When I had similar problem with dedupe, creating antivirus exceptions didn't help. I had to uninstall AV-program from the server. I'd recommend you to rule that option out.

PS. All of the above is guess work as only the BE developers know where the real bottlenecks are. But I really don't think it's in our hardware. I'm guessing that it's MD5 hash - could be a more effecient hash algorithm but I doubt Symantec will divulge that.

Cheers, Rob.

Anyone expecting really exceptional throughput with BE and deduplication is going to be disappointed IMO. We get around 800MB/min on a disk system that can easily handle 6,000MB/min.

This is due IMO to two fundamental bottlenecks in BE:

1. Speed that the agent (or media server) can calculate the hash values, e.g. MD5 (guess) is never a fast operation
2. Potential performance issues of the PostgreSQL database used to hold the deduplication database to look-up the hash values and write modified blocks

The remote agent on BE is single threaded and some non-scientific tests I carried out showed that our file server managed about 1000MB/min calculating MD5 hash values. So that is an immediate bottleneck on top of your file servers disk subsystem. You could have solid state disks in there but the throughput would be limited by the hash algorithm.

A solution is to make the agent multi-threaded and it's an ideal candidate if the disk system can supply data faster than a single core can hash it. It's naturally split into 64k blocks.

The bit about PostgreSQL performance is a bit of a guess but it's based upon the fact that you need 1GB of RAM per TB of data being backed up - I can't believe is needs so much for what is a very simple database requirement at it's heart - looking hash value and if not there, write the block. The overriding requirement here is performance.

Most competitors to BE make a big story of how much more efficient their deduplication engine is.

Cheers, Rob.

A.W. -

This is not a solution for all of the slow job rates, but we have recently identified that our R510 BE Server from October 2011 was misconfigured from the factory.  The misconfigure is more likely to happen with a single processor, but it might be good for you to review your riser card configuration vs. where your controller is for the hard disks and the nic(s).  It is possible to get severely degraded performance if the add-on cards are not placed into the correctly rated slot. (x4, x8, x16).