cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted

NDMP deduplication ratio is pool for MSDP

Hi All,

I am using EMC VNX, backup storage are DD and HP D3600 (MSDP). I found that the deduplication ratio is very pool which compared with  NDMP backup to DD and D3600.

Is it normal for MSDP keep EMC NDMP data? Pls. advise, many thanks.

Regards,

Ken

Tags (2)
7 Replies
Highlighted

Re: NDMP deduplication ratio is pool for MSDP

Hi Ken

As you have discovered NDMP backups to MSDP by default do not dedupe particularly well for NAS filers that do not have NetBackup stream handlers (such as NetApp). The reason is the default behavious for MSDP is to use a fixed block size when "chunking" the data. 

All is not lost though, there is a feature called VLD (variable length deduplication) whichyou can enable on the MSDP. With this set you should get similar deduplication rates as the DD.

I refer you to two links - one a previous answer in VOX from @Anshu_Pathak  https://vox.veritas.com/t5/NetBackup/MSDP-Variable-Length-Deduplication/td-p/863204

And an Veritas FAQ https://www.veritas.com/support/en_US/article.100042164?profileURL=https%3A%2F%2Fvam-profile.veritas...

These should help resolve your poor dedupe rates. 

Regards
D.

Highlighted

Re: NDMP deduplication ratio is pool for MSDP

Hi David,

Is there no way to increase the dedup ratio of existing backup image in HP D3600 ?

And If the NDMP image is copied from DD to MSDP, enable VLD is for not function well, right?

(due to DD is almost full, I copy the NDMP images to D3600 for offload the usage)

Regards,

Ken

Highlighted

Re: NDMP deduplication ratio is pool for MSDP

There is no way to change the existing dedupe ratio of backups already in the MSDP storage.

Using VLD will only help with new backups. Make sure you read the FAQs and also refer to the NetBackup Deduplication Guide, in particular the section "About variable-length deduplication on NetBackup clients".

Re: NDMP deduplication ratio is pool for MSDP

Thanks, but one more thing want to clarify.

if there are two backup image files from DD, the content is almost the same just the backup id is different.

When I copy those images from DD to D3600 (MSDP), how is the duplicate ratio for those 2 files in D3600?

Highlighted

Re: NDMP deduplication ratio is pool for MSDP

If the backup image ONLY differs in the name (aka backupid), then the second copy will deduplicate 100%. How much the first copy deduplicates is impossible to predict. 

If there is also some difference in the data within the backup it really is impossible to predict how much the second copy will dedupliate, but if VLD is being used, the dedupliction rate should be high; if VLD is not in use, then I would guess that the deduplication rate of the second copy would be poor (but dependant on where the difference between the two files starts).

Highlighted

Re: NDMP deduplication ratio is pool for MSDP

Just want to clarify that not only backup to MSDP , but image copy to MSDP pool will also has benifit for duplicate ratio.

Thanks.

Highlighted

Re: NDMP deduplication ratio is pool for MSDP

I don't know the answer to that but my guess would be it does not. However generally speaking using two different types of deduplication (DD and MSDP) is not a good idea; at the very least for duplicating images you have to re-inflate, copy everything and re-run deduplication processing.

There is a NDMP stream handler for MSDP for VNX (see HCL NDMP section) but I believe it is only called for backup - it's not required if you duplicate to MSDP which is the normal use case. You may want to backup as opposed to duplicate some VNX NDMP backups to MSDP to test what deduplication ratios the stream hander gives you.

Andrew