cancel
Showing results for 
Search instead for 
Did you mean: 

NBU 7 NDMP Deduplication Question

jsucram
Level 2
Hello everyone ..

I have recently upgraded to NBU 7 and implemented media server dedup .  I am very impressed overall with the functionality but the dedup rates for NDMP backups are beyond poor.

Has anyone else experienced poor dedup rates on NDMP backups on PureDisk?

My NAS device is an EMC Celerra and backup policy include these directives

set type = tar
set direct = yes
set option = NT
set snapsure =y

Any suggustions ... comments?
1 ACCEPTED SOLUTION

Accepted Solutions

Christoph_Linde
Level 5
Employee
Yes, dedupe on NDMP is not good. The problem is atm, that we do not know anything about the data inside the NDMP stream. The NDMP Stream is created by the NDMP server and the format is pretty proprietary to every NDMP server.

But Symantec is  working on a solution for this problem. I can not give any details or timeframes. Please be patient.

View solution in original post

8 REPLIES 8

Christoph_Linde
Level 5
Employee
Yes, dedupe on NDMP is not good. The problem is atm, that we do not know anything about the data inside the NDMP stream. The NDMP Stream is created by the NDMP server and the format is pretty proprietary to every NDMP server.

But Symantec is  working on a solution for this problem. I can not give any details or timeframes. Please be patient.

Mouse
Moderator
Moderator
Partner    VIP    Accredited Certified
Just an assumption: IIRC Celerra has an option to perform deduplication of stored data. Maybe your backup is deduplicated already?

jsucram
Level 2
Thanks for the info.  I'll hold off on sending NDMP data to a puredisk pool for now.

Mouse - unfortunately we aren't licensed for dedup on the Celerra.  I'll probably just keep alot of checkpoints to offset it not deduping well.

jim_dalton
Level 6

But it worries me that you say the data stream is proprietary to every ndmp server.

Surely thats the whole point of ndmp, its a protocol, it adheres to the standards, so every ndmp stream should conform to ndmp standard, otherwise its not ndmp.

If you know differently, thats a concern.

Tim

Ed_Wilts
Level 6
NDMP is a communications protocol - it's not a data format protocol.  NetBackup knows how to tell a NetApp filer to dump its data to tape, but it doesn't tell the filer exactly how that data is to be written.  That's how you can get vendor-specific metadata into your backups without Symantec having to write custom code to support each filer (like it has to do for Linux, Solaris, VMS, etc.).

This is also why you can't tell a NetApp filer to create a backup and then restore it on a Celerra.

TechTarget has a definition of NDMP here:  http://searchstorage.techtarget.com/sDefinition/0,,sid5_gci837192,00.html

From the FAQ on ndmp.org (http://www.ndmp.org/info/faq.shtml):  "In theory NDMP doesn't care about tape formats. The NDMP DATA server produces an NDMP stream and this is simply written straight to tape by the NDMP TAPE server."

Christoph_Linde
Level 5
Employee
Tim,

what ed wrote is correct. To say it in simple words: If a NDMP vendor writes a tar, a dump, or something completely proprietary is up to the vendor. Also if the vendore is using tar, it does not mean in any way, that u can restore it with a standard tar.

The NDMP protocoll is telling the NDMP Filer: "I want you to backup this filesystem". It also tells the NDMP filer to do it to an attached tape, or over the network to another NDMP host. Also the filer transfers meta informations to the backup server. Thats it.

So the communication protocol is defined, the content of the backup stream is not.

Compare it to http protocol. The communication is defined, the content of a file download for instance can be everything, http does not care.

teiva-boy
Level 6
Try and restore an NDMP backup to an EMC NAS, or an EMC NDMP backup to an Isilon NAS.  NOT POSSIBLE.

Just as stated, while there is a standard for the PROTOCOL, each vendor implements it differently and each vendor does not implement the same features as part of the NDMP feature set.



That said, NBU 6.5 with PureDisk 6.6 and/or NBU7 will be getting something for you NetApp shops soon according to a Symantec rep....

jim_dalton
Level 6

That has some truly unpleasant implications and Im glad I know it! Thanks.