Occasionally slow backup on exchange server with BE12.5

[cubic_nz]

Hi guys,

I am hoping someone will be able to point me in the right direction to troubleshoot some odd behaviour I am seeing with our exchange server backup.  Sometimes the backup is very slow and ends up being cancelled because it exceeds the maximum time allowed for the job.  Other times it is normal speed and finishes with half the time to spare.  I'm not sure what info is most useful so let me know if I need to provide more info:
Job size around 16GB
Time allowed 5 hours
When the job goes fast it takes less than 2 hours, often takes 4 hours plus, sometimes more than 5 hours
The server is about 6-7 years old, running exchange, SBS, backup exec 12.5
The tape drive is an HP StorageWorks Ultrium 448, with a SCSI conneciton I believe.
I have 3 back up jobs, daily, weekly and monthly, and they all use the same selection list and settings, they just run at different times
The weekly job runs on Friday mornings and is always the fastest (slowest backup averaged 145MB/min)
The daily job runs in the middle of the night and is slower (fastest is 74MB/min)
The monthly job runs in the afternoon once a month, and is usually fast but today was very slow (14MB/min)
We are quite a small company with only about 12 computers in our office and I have no formal training in computer administration, just figure it out as I go!

I cancelled today's monthly backup after 4 hours (4GB complete), rebooted the server and tried again without changing any settings or the tape and it completed in 2 hours (16GB).  On the first attempt I noticed that the activity light on the tape drive would flash to show activity for a few seconds, then go solid for no activity for a few seconds, then flash again, etc.  I watched it a little while on the second attempt and it started off really fast at over 200MB/min, then slowed to a crawl in the middle, and then picked up again at the end.

My guess is that it is struggling to backup the exchange server files because they are in use all the time.  However, they were still in use on the second attempt and that doesn't explain why it's slower at night when activity would be negligible and fast during the day when everyone is in!

As I said, I have no training in this and it's easy to get confused with all the acronyms and jargon.  Can anyone suggest (in plain english!) what I can do to try and diagnose the problem?

[CraigV]

Hi,

Check the following:

1. No antivirus scans being run during the backup window. If an AV is running, there is always the chance it will interfere with your backups; it would affect throughput while a backup file is being scanned for example.

2. No maintenance is being performed (ie. Exchange maintenance) during the backup window.

What happens if you run your backups at night? Generally you would run your backups a t a time where there is minimal interference on the server...users aren't logged on and accessing files/Exchange/SQL etc.

[cubic_nz]

Hi Craig,

That may have been a factor yesterday, as the virus scan ran through until 10am, but the backup was still very slow at 12pm.  That also won't affect the daily backups as the full virus scan is run only once a week on Sundays.  I've set the weekly virus scan to start 6 hours earlier, at midday Sunday, so it will be well finished by the time we start on Monday, just to be extra sure.

Definitely no maintenance being performed by me.  Are there any automated maintenance tasks that I should check for?

It's the night time backups that are the problem!  The day time backups are usually ok.  At night there are no users logged on, and exchange server would have very minimal incoming emails.

Last night I put my weekly backup tape in for the daily backup.  Since the weekly is usually good I thought it might be a worn out tape issue.  Afraid not, the weekly backup tape behaved exactly like the daily tapes do, the job finished just before the deadline, 4h:55m, compared to a normal time for that tape of 2h:20m.

The automated maintenance idea sounds promising, are there any exchange server maintenance tasks that typically run at night?

[CraigV · Accepted solution]

The link below might shed some light on the Exchange maintenance tasks:

http://www.goexchange.com/faq/faq_GEvsMStools2.asp

I have also seen that when an antivirus scans the BE installation directory, it can cause issues with it. You might want to exclude that (for now...just to eliminate it), and the AOFO cache directory.

[cubic_nz]

Hi Craig,

I think I've worked it out.  Last night I shifted the backup start time from 11pm to 3am and the time taken reduced by 3 hours.  From reading the link you gave I've checked the application logs and worked out that at 1am exchange defrags the main database.  When this runs concurrently with the backup it took 3 hours, and slowed the backup over that period.  When the backup wasn't running it took 1h:40.

Not sure what slowed the backup on Friday morning, maybe more digging in the event log will solve that, but at least it looks like my nightly backups will become reliable again!

[CraigV]

Cool man...glad it helped, and I hope it sorts out the rest of your backups. There is nothing worse than having to trawl through possible issues to try fix a problem.