cancel
Showing results for 
Search instead for 
Did you mean: 

Slow performace from BKExec 2012 to DeDuplicated Disk

leeweedon
Level 5

Hello,

We've just moved to BKExec 2012 from 2010. Fully removing then installing the software from scratch.

We have two BKExec servers at different sites, both the same hardware spec, Windows 2008 X64 R2, SP1 with 32 GB memory.

Our deduplication area is hosted on or Equalogic SAN connected by the LAN direct to the Backup Exec Server.

The problem we are having is that backup are slow, especially exchange. Backing up a 171gb information store took almost 8 hours and failed reporting '-1022 There is a disk IO error'

The backups speeds initially build up to around 900MB/s, but soon fall down to 390MB/s.

Backing up a file server (1.5TB) took 32 hours at a rate of 778MB/s

Performance wise, network on both server (backup and target) is never higher than 25% for both the network and san links. Memory usage is about 40% with cpu hardly being used.

With the file server, we have F-Secure, but we ran a pre and post job command to stop this from scanning while the backup was running.

Windows and BackUp Exec are both fully up to date.

Any help on this would be greatly received.

Thanks

Lee

 

17 REPLIES 17

Jaydeep_S
Level 6
Employee Accredited Certified

Performance issues can be tricky as it might be a combination of lots of potential causes. Please look through the article below as they are fairly elaborative about what could be done.
http://www.symantec.com/docs/TECH49521
http://www.symantec.com/docs/DOC5481

However, I would suggest you to to try to disable Check Point Restart. Backup Exec 2012 enables the Checkpoint Restart feature by default.  This can make the backup speed slower than previous versions
http://www.symantec.com/docs/TECH191072

Javed_Nawaz
Level 3
Employee Accredited Certified

Hi leeweedon,

Any update from the last Engineers posting on your issue ?

 

CraigV
Moderator
Moderator
Partner    VIP    Accredited

Hi,

 

1. Yesterday was the 25th...give the OP a chance to test!

2. Please stop sending update requests in the forums...rather send a PM! Nothing new to add by doing so.

Thanks!

leeweedon
Level 5

Thanks for the reply Jaydeep.

We have identified the problem to be the backups of the Information Stores. A flat file backup of the exchange server works at around 1000MB/SEC which is not ideal, but acceptable.

When backing up the IS using the agent it still runs less that 300MB/sec   We have tried disabling Verify, Test Run, Enable Checkpoint Restart but this doesn't change anything.

I've even tried without the consistancy checks being enabled, but still no improvement.

 

Our exchange server 2007 is running Windows Defender and ForeFront.

 

CraigV
Moderator
Moderator
Partner    VIP    Accredited

...is your AV perhaps scanning the dedupe folder?

leeweedon
Level 5

Do you mean the AV on the BKEXEC server? If so, no. I removed the AV completely from it.

CraigV
Moderator
Moderator
Partner    VIP    Accredited

That's what I meant Lee. Is the AV on the Exchange server perhaps scanning the beremote.exe service? What happens to your backup of the Information Store if you disable the AV on the Exchange server?

Also, what is the configuration of your storage on which the dedupe folder is located?

leeweedon
Level 5

I have just tested using a pre-job command to stop Windows Defender. This seems to make no difference, but I will test this fully this evening.

Jaydeep_S
Level 6
Employee Accredited Certified

leeweedon could you please try and run the Information store job to a test B2D folder or a Tape Device,instead of Deduplication folder/device. This will atleast isolate, if it is the transfer from source or th write to target is in fault.

Also would be nice if you can try the steps in http://www.symantec.com/docs/TECH186741

leeweedon
Level 5

Carrying on from this it doesn't appear to be AV related. 

Backing up to Local Dis Storage (non-DeDupe) is also very slow.

CraigV
Moderator
Moderator
Partner    VIP    Accredited

OK...let's go back to the start...you have 2 backup servers on separate sites. Are you backing up across a WAN link to your dedupe folder?

leeweedon
Level 5

We do have two backup server, one at each site. The sites are connected by a 1GB LES. 

The backups we currently have setup do no go accross the sites, they are done by our server at site A to a de-dupe folder on SAN based on Site A.

CraigV
Moderator
Moderator
Partner    VIP    Accredited

OK great...so no WAN bottleneck with servers on Site B being backed up to Site A.

So if the backup to a normal B2D is slow, this rules out dedupe. The AV is ruled out as it isn't running.

What are the specs of your backup server? Are all your NICs and the switch ports they are connected too hard-coded to the fastest speed of the NIC? ie. 1000MB FULL?

Have you checked for any hardware failures, or hardware in prefailure?

Thanks!

leeweedon
Level 5

The NIC has been configured in a team, dual 1000mbps, connecting to our Cisco core (both configured in a port-channel) 

I've got the latest drivers etc and all hardware apears to be working ok.

The only bottlenick I can think of is possibley local HDD read/write IO?

 

BK Exec servers are 64bit, 32GB RAM, Intel Xeon X5550 DUAL x QUAD CORE @2.67

Windows 7 Server 3008 R2 Standard.

leeweedon
Level 5

Backups from any server are running much better now we have teamed the NICs.

However, Exchange is still a killer. averaging 600MB/min.

 

Forefront security is running, but have tried to fully disable this but it makes no difference

Gurvinder
Moderator
Moderator
Employee Accredited Certified

Recommend you to open a case with tech support. Also does this job fails aways ? and have you seen the Issue with both Full and Incremental to dedupe ?

pkh
Moderator
Moderator
   VIP    Certified

.Did you realise that the last answer to this discussion is almost 2 months old?