BE2010 R3 / VMware Agent: Backups get slower the longer a job runs

[xadox]

We are backing up 4 Windows Server 2003 VMs with GRT set to on. The mashines are on a Dell MD3000i SAN.

While the backup starts with realy goot speed arount 3500MB/min it drops to around 100MB/min at the middle ot the whole backup.
The target device is a regular B2D Folder on a RAID5 (not a dedupe store).

[AmolB]

GRT backups do take long time to complete. Have you checked the fragmentation level of the disk 

which hosts B2D folder.

[marcymarc24]

Is the byte count actually increasing?

We have a similar issue, backing up to a disk duplication folder, the backup just grinds to a halt mid backup and hangs.

Is this a bug with 2010 R3?

[xadox]

@AmolZeroCool

It is a 4TB RAID5 with about 3,5TB of free space on it. The filesystem is also defragmented.

@marcymarc24

Yes the byte count is increasing.

[zalewskim]

We have a very similar set here;

Dell 2950 8Gb Ram, Xeon E5335 2Ghz CPU connected to a Dell TL2000 via SCSI and connected to a Dell MD3000i with 14Tb SATA drives via ISCSI

We hvae the same issue when backing up VMs or Physical machines the speed fluctuates massivly in terms of job rate. The backup speed could be 200Mb/min -> 1200Mb/Min the speed seems to change by the end of the job.

We have got GRT enabled on the jobs and I have made sure that everything is upto date on all the hardware. The backup to tape jobs seem to run at 500Mb/Min which is very slow! If I was to back up to tape from the local server it will run about 2000Mb/Min.

As soon as anything goes to the MD3000i it goes very slow, we are running a HP Procurve switch network and I have monitored all the network ports for all equipment while the jobs were running and the ports are around 30% used. So we have loads of capacity on the network.

[Sublaner]

We saw something similar and infrequently, on a smaler scale, however testing Duplicate jobs to iSCSI targets (Iomega StorCenter ix2 and Netgear ReadyNAS Pro) through a 1Gb port. For Duplicate jobs GRT should not enter into it (decreasing speed) though the Duplicate jobs do run faster than the GRT Full jobs. Running multiple jobs at once there seems to be some sort of prioritization going on, but even with multiple jobs going I seldom saw port utilization go over 25%. With one of the devices the jobs would start quickly and taper off, and slower jobs did not neccessarily pick up speed as smaller or faster jobs completed. We saw a clear performance advantage with Netgear (1833 vs. 827 MB/min), suggesting something in the target configuration or hardware was responsible.

Anyway for what its worth, Microsoft (http://technet.microsoft.com/en-us/library/ee338480(WS.10).aspx) lists some iSCSI best practices for Networking that includes Disable unicast storm control, use MPIO, enable flow control on network switches and adapters, and Turn off spanning tree. If any of that helps with your ProCurve, I'd be interested to hear about it here. Cheers,

[xadox]

Any new informations on this problem?

I also have the problem while backup up files with RAWS. Starting with about 2000MB/min it gets down to 10MB/min.