cancel
Showing results for 
Search instead for 
Did you mean: 

BE2012 - Duplicate from DeDup Device is VERY slow

OP
Level 4
Partner Accredited

Hopefully someone can help me - I'm running out of ideas:

Problem:

We're using DeDuplication for the firststage-backup, afterwards the data is copied to external media - this time USB-disks (USB3 with USB3-controller)
The first stage is running fine, but the copy to usb is awfully slow - and gets slower every day.

It started with about 1400 MB/min, now it's down to 150-250 Mb/min.

To have a comparison to a "normal" duplicate to usb-job, I've created a b2d-device on the same storage as the dedup-foleder, made a backup and duplicated that to usb-disk - everthing identical, except the b2d-device used as a source instead of the dedup-device.

The result is about 6.000-7.500 MB/min - so the diskchannel and the usb-disk-channel is fine.

I don't have a problem with the slower speed from the dedup-device (due to the rehydrating) - as long it stays above 1000 MB/min, but it mustn't drop that far.

Environment:

BE2012 with actual updates. The hardware is a HP DL180G6, CPU: E5606, 12 GB RAM. 5 SAS 1 TB Harddisks attached to an HP SmartArray P410 with 256 MB Cache and battery. CPU usage below 50%, RAM-usage ~ 50%.

 

Any idea ?

Thx

Olaf

1 ACCEPTED SOLUTION

Accepted Solutions

OP
Level 4
Partner Accredited

Hi folks!

I just wanted to update and close this case.

To make a very long story short:
According to the 1st, 2nd and 3rd level supportengineers from Symantec  we do not have a problem - the dedup works like designed and they can NOT do anything to speed up the process of dplicating FROM the dedup to any external non-dedup-device. :-((

Due to this sad result and the amount of time we've invested in this case we've decided to discard dedup as a primary backup target at our and all customer site's.

From my point of view the product can't keep the expectations from the customers and promisses made by symantec.

 

CU

Olaf

View solution in original post

19 REPLIES 19

CraigV
Moderator
Moderator
Partner    VIP    Accredited

Unless you're duplicating to another dedupe folder, you should actually be rehydrating your data...this is 1 reason for slower speeds. Add in a USB drive and you would have the right environment for slow duplication rates.

The server hardware itself looks to be fine, with no changes to be made!

Thanks!

OP
Level 4
Partner Accredited

The absolut speed of the (beginning) dedup rate isn't the problem.

THe quesion is - why is it getting slower every day?

Without any visible bottleneck regarding disk, cpu or ram ?

Imho that's not the way it should be.

Olaf

CraigV
Moderator
Moderator
Partner    VIP    Accredited

Have you used Perfmon.exe to see whether or not there is a bottleneck on that USB drive?

Thanks!

OP
Level 4
Partner Accredited

I'm pretty sure, that the usb-disk is not the botteneck due to 2 facts:

- Copy from B2D to USB works with ~ 7.000 MB/min

- Copy from DeDup to USB worked in the past with more than 1.500 MB/min.

I haven't used perfmon but the ressourcemonitor in the w2k8 task manager and couldn't see any problems.

 

Regards

Olaf

AshutoshTamhank
Level 4
Employee Accredited

Is that a GRT / non GRT enabled backup you are duplicating?

CraigV
Moderator
Moderator
Partner    VIP    Accredited

...please edit your previous post if you post within minutes of the last one!

Thanks!

AshutoshTamhank
Level 4
Employee Accredited

As a test what happens when you do a duplicate from your dedupe back to the dedupe folder?

Is that a GRT / non GRT enabled backup you are duplicating?

AshutoshTamhank
Level 4
Employee Accredited

You seem to read my mind!

OP
Level 4
Partner Accredited

Its a duplicate of a backup from a w2k8-server - raws based.

It includes Exchange-Data/mailboxes - so the answer is yes and no.

Does that make any difference ?

AshutoshTamhank
Level 4
Employee Accredited

When a backupset is duplicated from a dedupe device to a non dedupe device, data is rehydrated i.e reconstructed. The metadata(catalogs) also has to be rebuilt. GRT backups are catalog intensive than a non GRT backup. To isolate the issue, I would suggest you to create a backup without GRT and test a duplicate backup. I assume there are no other operations running on the dedupe folder during the duplicate backup. 

OP
Level 4
Partner Accredited

Hi all!

a few answers / comments to the suggestions made by you:

1. Yesterday (out of the blue sky) the dup-job ran with unbelievable 1700 MB/min - completly unexpected, because it was the same job / usb-disk / data that was duplicated.

The one and only difference was:
I started the duplicate-job manually with "run now" from the serverconsole itself - not via rdp.

So imho - there is no connection betwenn grt/non-grt data and the dup-speed.

2. Today - same dup job starts automatically at 9 am - and it's running with only 223 MB/min :(

Changes since then:
I've installed last night the patch 201596 and restarted the server and today it's a different (identical type) usb-disc.

I'm gonna check if there is a difference when:

- The job is started manualy directly on the console without rdp (it shouldn't...)
- The disc from yesterday is used

OP
Level 4
Partner Accredited

New facts:

1. Starting the job manually from the serverconsole directly didn't change a thing - still veeeeeeeeery slow.

2. Using a brandnew empty freshly formatted usb-disc afterwords - even slower - 180 MB/min

3. Restarting the be-services before restarting the same job - juhu - speed is up to 1126 MB/min - still accelerating.

Some conclusions:

1. there seem's to be no connection between the usb-disc itself and the speed of the backup

2. the "BackupExec-setup" itself (hardware, software, configuration) seems to be ok, as the system is able to perform the duplicate-job with high-speed

Do you agree with ?

But why change a restart of the be-services speed up the duplicate job ??


Waiting for some ideas..

Olaf

CraigV
Moderator
Moderator
Partner    VIP    Accredited

...restarting the BE services is obviously releasing "something".

My suggestions:

1. Check that no AV is actively scanning the dedupe folder, and if so, put in an exclusion for that.

2. Check to see if there is a Known Issue in that section around this:

https://www-secure.symantec.com/connect/backup-and-recovery/issues

Thanks!

OP
Level 4
Partner Accredited

Craig,

I've got the same impression - it's pretty obvious.

The AV check exclusion is already done - and not modified between the diferrent tests / results, so that couldn't be the reason for different behaviour.

I couldn't find a similar problem / error in the list.

Regards

Olaf

OP
Level 4
Partner Accredited

I've opened a case with symantec.

I'll keep this post updated.

Olaf

 

CraigV
Moderator
Moderator
Partner    VIP    Accredited

Hi Olaf,

 

In connection with my post on the link below, if you updated BE 2012 which caused the issues you now face, see if you can uninstall those patches and make sure you can operate as-per-normal. Once done, install the patches one-by-one to test...

https://www-secure.symantec.com/connect/forums/problems-deduplication-2010-r3?page=0#comment-8417931

Thanks!

OP
Level 4
Partner Accredited

Statusupdate:

Case is opened with Symantec TechSupp - they are working on it - already escalated to the next level.

OP
Level 4
Partner Accredited

Just a little update:

The case is opened for several weeks now, but Symantec is totally ignoring the facts and sticks to the statement "rehydrating makes duplicate from dedup-devices slow".

But the case is not closed yet.

 

OP
Level 4
Partner Accredited

Hi folks!

I just wanted to update and close this case.

To make a very long story short:
According to the 1st, 2nd and 3rd level supportengineers from Symantec  we do not have a problem - the dedup works like designed and they can NOT do anything to speed up the process of dplicating FROM the dedup to any external non-dedup-device. :-((

Due to this sad result and the amount of time we've invested in this case we've decided to discard dedup as a primary backup target at our and all customer site's.

From my point of view the product can't keep the expectations from the customers and promisses made by symantec.

 

CU

Olaf