Forum Discussion

Shyam_Prasad's avatar
Shyam_Prasad
Level 4
16 years ago

Data Deduplication!!

Hi,

What is meant by De-duplication?
Is De-duplication enabled by default in NetBackup?
If  not how to goahead for De-duplication?
Pls let me know..

--Jaya
7.1.x and Earlier
Backup and Recovery
NetBackup
  • Seth_Bokelman's avatar
    Seth_Bokelman
    16 years ago
    De-duplication is the elimination of redundant data from backup media.  It applies mostly to backups performed to disk systems, not tape systems, and it's not a feature of NetBackup, per se, though I believe you can do it with Symantec's PureDisk product.  Typically, people who are using VTL devices, such as Data Domain or NetApp, will use DeDupe so that they can store the equivalent of 200 TB of backup data on 20 TB of disk.  So, you could attach one of these devices to your NetBackup installation, and then you would have deduplication with NetBackup.

    Basically, it works by breaking all of your data into chunks, performing a hash of each of those chunks and looking for hashes that are identical.  The systems then compare the data, and if it's truely identical, they'll only keep one copy and throw away they others.  As this is all happening on the storage shelf level, it's transparent to your backup software, and the space savings are massive, as backup data often doesn't change much between fulls...

    That's a quick 'n dirty explanation, I'd also suggest the Wikipedia article: en.wikipedia.org/wiki/Data_deduplication

  • Seth_Bokelman's avatar
    Seth_Bokelman
    Level 5
    16 years ago
    De-duplication is the elimination of redundant data from backup media.  It applies mostly to backups performed to disk systems, not tape systems, and it's not a feature of NetBackup, per se, though I believe you can do it with Symantec's PureDisk product.  Typically, people who are using VTL devices, such as Data Domain or NetApp, will use DeDupe so that they can store the equivalent of 200 TB of backup data on 20 TB of disk.  So, you could attach one of these devices to your NetBackup installation, and then you would have deduplication with NetBackup.

    Basically, it works by breaking all of your data into chunks, performing a hash of each of those chunks and looking for hashes that are identical.  The systems then compare the data, and if it's truely identical, they'll only keep one copy and throw away they others.  As this is all happening on the storage shelf level, it's transparent to your backup software, and the space savings are massive, as backup data often doesn't change much between fulls...

    That's a quick 'n dirty explanation, I'd also suggest the Wikipedia article: en.wikipedia.org/wiki/Data_deduplication