Deduplication : PureDisk software appliance vs 5020 hardware appliance

I compared the 5020 appliance to just a few shelves of Dumb HP SAS arrays...The HP arrays blew away the 5020. I am sure it is the 6gb SAS performance that did it.

Don't worry - I don't blame you at all for questioning how we can take on the big guys on performance and win, and I appreciate the opportunity to respond!

And I also kow that Symantec support is improving as of late.  If you have any issues please let them know so they can address it.  

Anyone from support listening to this discussion please jump in!

I know I'm giving you a hardtime, but I will say this.  I applaud that you give support above and beyond Symantec's own.  Symantec's own support is HORRIBLE!  L2 or an escalation manager is the first thing I ask for, even then it's days to weeks to solve.  I've had cases open for months too!

I have installed two of the SD3 units into our environment, the level of support and dedication that InQuinox places behind the SD3 appliances is by far the best I have ever experienced. Ray worked with me personally on many occasions to resolve an issue when Symantec would provide less than adequate support. The performance numbers are not fluff or "Marketing" but real numbers, the InQuinox SD3 deduplication appliances are fast if not the fastest and the way the company as a whole stands behind its products is second to none. If you are in the market for a deduplication appliance don't overlook InQuinox in your search they will not disappoint.

Kevin Polk

It's PDDO through and through.  Rehydration is faster on the SD3 because of three optimizations: 

1.  The metadata is stored on disk optimized for high IOPs.  This allows PureDisk to look up the references to the data segments constitutiong the data stream very very quickly.

2.  Once the location of the data segments are known, they must be retrieved from disk as fast as possible.  This operation requires the content data segments be retrieved from a disk subsystem tuned for high throughput.

3.  Finally, the rehydrated data needs to be sent as fast as possible to the Media Servers.  We use dual 10Gb ethernet bonded interfaces between the PudeDisk nodes and the Media Servers so that the rehydrated data streams can have a large enough pipe to stream multiple tapes.

Hi Ray,

Are you quoting rates based on natively backing up or used with Netbackup in the same manner as Puredisk doing PDDO?

I'd be interested in best/worst case type statistics for rehydration to LTO5 tapes when used with Netbackup.

teiva-boy,

The numbers are based on deduplicated data streams - that's the point. They are real data streams being stored in the appliance. It cannot digest 1 TB/min of raw data, but you will never send raw data - by that I mean random data that cannot be deduplicated - to a deduplication appliance. We used a dedupe rate of about 15x, which is what DataDomain uses for their published deduplication rate.

At that rate, we can handily digest 1 TB/minute. Raw disk speed is phenomenal on our SD3. On a 6 node (192 TB) SD3 we can write (direct to disk) a little over 5 GB/sec That's 833 MB/sec per node compared to HYDRAstor's 750 MB/sec per node. But I assure you, the tuning does not end with raw disk speed. We carefully configure the disks to optimize throughput specifically for the Content Router data and we optimize IOPs for the metadata.

The reason you don't want to compare the ingest of non-dedupable data in the different solutions (NEC's HYDRAstor, EMC's DataDomain) is because it isn't useful. RAW IO is not proportional to dedupelication ingest rate. You want to measure deduplication input because that is how it will be used and this is a direct measure of the performance of the entire system. PureDisk excels at this once you put it on high performance, tuned hardware. If you look at CPU Usage on the SPA and the nodes during ingestion of data you will see a completely different footprint with non-dedupable data than you will with data that is being deduplicated. PureDisk ramps up far more quickly than the competition as you vary the dedupe rate - because of how it hashes the fingerprints on the Media Servers, as well as many other accelerators.

Hope that helps!

The 1 TB/min ingest rate of the SD3 is based on a deduplication rate of about 15x.  From what we can gather, this is the rate that is used by DataDomain when they publish their 23 TB/hr rate, so this rate is a fair comparison.  Ingest rate on raw data is handled differently - it has a much different, less efficient CPU profile and is not a good measure of the ultimate performance.  You are only going to get raw data ingest rate on the first backup of new data - and after that there will be duplicates. What I mean to say is that the speed at which a system can ingest raw data is only one measure and it may not be the best measurement.  You have to measure the ability of the dedupe appliance to fingerprint the data stream, recognize duplicate data, and store the new data. The processing is different.  When you store never-seen-before unique data into PureDisk, you will see a CPU performance profile that differs greatly than if you are sending data that has been seen before. Most backup data being sent to the appliance has been seen before.

That said, speed of the disk is only the first part of achieving a fast ingest rate.  We use a proprietary method to ensure that the Content data can be written as fast as possible on the disks (high throughput), and that the metadata is on a disk subsystem capable of high IOPS. Once that is done, re-hydration and backup performance dramatically increase. We've done a lot of work to achieve that 1 TB/min ingest rate.  I would welcome a test against anyone - I promise we'll blow them away and then they can all go home.  :-)

1TB/min?  Marketing numbers much?  You can't use client-side dedupe numbers to factor that value.  You need to use the ingest on raw data, or using target-side dedupe.  

I'm sure your "numbers," are accurate, but at the same time, misleading as the box can't really ingest 1TB/min of actual data, you're assuming a dedupe rate of 10x or greater at the client.

If your numbers are true, the NEC HydraStor (claimed fastest in the world), EMC DataDomain's (fastest claimed in north america, they seem to not count NEC in Asia LOL) and Exagrids (Their grid architecture is neat) of the world should just give up on their game.

InQuinox SD3 appliances are available as high-end models as well as low end models. The high end models provide High Availabliity, built-in 10 GB Switches, high-performance media servers, and self-healing, high perfromace disk subsystems (using fault detection isolation and repair technology) with rapid rebuilds desined to last years without drive replacements.

The mid-range systems are available as MSDP servers that can also function as FT Media Servers, and as Windows Backup Hosts for VMware, and can optionally house a SanXfer Library server to provide SanXfer capabilty for a degree of data protection not availble anywhere else.