Showing results for 
Search instead for 
Did you mean: 

Duplicate Backup Sets between Deduplication Storage using client-side deduplication

Hi there

I'm currently Trialing BE 2010 R3 and in My scenario I have two sites that both have a Media Server with Deduplication storage attached and I'm in a CASO environment with the CASO server in Site A and the Managed Media Server in Site B.

I've created a backup job to backup a server that is around 23GB to the deduplication storage on the CASO Server in Site A and also created a Duplicate Set job (configured to run after job) to duplicate that backup set to the deduplication storage on the managed media server in site B.

In the Duplicate Job settings I selected the 'Use client-side deduplication' as I wanted to reduce the amount of data being transferred across the remote link on a nightly basis.

the initial job ran fine and took 30 minutes to backup locally (1GB Network connection) and then a further 1hour 30minutes (100MB WAN Link simulated with Firewall) to duplicate that job to the Media Sever in Site B.

I ran the job a second time and again it took the same amount of time to backup to the local media server's deduplication storage (because I'm using media server deduplication) and we had a nice healthy deduplication rate of 33.0:1, however the duplicate job still took 1 hour and 30 minutes and from what I could see it was again copying the full 23Gb across the WAN link. When the job finished It did show a nice healthy deduplication rate of 33.0:1 on the managed media Server but what I'm trying to do is reduce the amount of data the duplicate set job sends across the link.

I went back into review settings on the duplicate job and noticed that the job settings had reverted to the 'Use media server deduplication' option that would explain why it took so long to duplicate the second backup job as in that situation, the deduplication would be done on the Media server in Site B. I tried changing this settings back to 'use client side deduplication' and submitted the job but everytime i reviewed the settings, they had reverted back.

Either it's not a supported option and the system isn't telling me that when I select it or I'm doing something else wrong. I reviewed various options and shared all the deduplication folders between servers etc.. but cannot figure it out.

I know I can target backups to send direct to the remote Media Server using 'client-side deduplication' but I thought this would be a far better way of doing things so I could backup a server once to local storage and then duplicate that local job across the WAN for DR purposes and duplicate locally to Tape for Archival purposes.

Can anyone advise whether what I'm trying to do will work?




5 Replies

I've made some progress on

I've made some progress on this issue.

It seems there is a bug in the software where it will set the duplicate sets job to use whatever de-duplication method is set in the Job Defaults under Tools - Options.

Even if you try to change it while creating the duplicate sets job or amending afterwards, it won't allow you to change it.

So you need to delete the job, change the job defaults and re-create the duplicate sets job

Obviously it needs fixing but at least I have a workaround and can confirm that what I want the software to do will work.





You are correct in the fact

You are correct in the fact that Media server to Media server deduplication does NOT support Client-Side Deduplication. The option will always revert to and use media server deduplication wen using opt-dupe.

Why not client-side on media servers????

Have been long time BE customer. We recently made a decision to get rid of tape backups, and ensure ourselves against fire protection by adding a second BE server (CASO option), at a colo site we have, across a WAN MPLS circuit. This was done with the thought that the dedupe option would help us in the sense that:

1 server would use BE to backup, to local attached storage, all the resources amongst agents on its local network

The 2nd CASO BE server (at colo site) would then use dedupe to backup those 'backup files' from the 1st server.

It makes no sense that media servers aren't allowed to use client-side duplication to talk to each other, since in so many cases, media servers are separated by these WANS.

Can you make a recommendation on how to structure my jobs? I already went ahead and made the change so that the CASO server (at the colo) is communicating directly with the agents whose resources I want backed up. Is there anything I can do with the structuring of 'full backups' and 'differentials'? Right now, I'm using a policy that says:

differentials Mon thru Friday night
fulls on Sat
verify on Sun

Last question: Is the displayed 'byte count', the amount of data that has been 'transmitted'???

It makes no sense that media

It makes no sense that media servers aren't allowed to use client-side duplication to talk to each other, since in so many cases, media servers are separated by these WANS.   

Client side deduplication uses a cache comprison processes to determine what fingerprints the client needs to send.

Opt-dupe uses a different method to compare the dedupe'd data in the source server to the destination server.

In my opinion, Opt-dupe is more accurate and faster to compare the needed data to send to the remote server. Using client-side between media servers would slow the whole process down.

This is all done through the deduplication services and is not transparent to the user or the BE UI.

Can you make a recommendation on how to structure my jobs?

In your scenario, you will want to use Opt-Dupe. Simply add a Duplicate template linked to your backup jobs to send the data to your colo.

Allow your local media server to do all the heavy lifting. All the backup jobs should be created on the local server and not delegated from the CASO. This will reduce network traffic on your WAN.

kudos to you

I had no idea what opt-dupe was, but after reading this:

It was pretty straight forward.  I've notice SUPER deduplication rates on the 'duplicate' template I added to my policy on the managed media server.  This 'duplicate' template runs after my 'backup' template, and has the source and destination devices set as the respective deduplication folders.

Thank you again.  Lastly, could you comment on how to structure my 'verify' template.  I was thinking about allowing the 'backup' template to verify after completion (since its backing up only local agents).  And then scheduling a 'verify' template on the weekends for my offsite duplicate data.  Again, this would be configured from the policy on the managed media server.  Is there anyway to configure a 'verify' template/operation on my duplicate jobs from the offsite CASO server, instead of the managed media server?  Obviously, I'm trying to ensure that this can be done over the WAN, so Im trying to avoid any scenario where I have upwards of 20 Gb (actual physical data) trying to get across the circuit.  I'n not sure how the 'verify' actually works ( much like I wasn't with the dedupe ) so any pointing in the right direction, would be greatly obliged!