Verify issue: Verify duration every second week 4 hours longer than the other week

[aos-stade]

Hi.

Since several weeks we have a strange issue with the verify of the full backup of our fileserver.

The full backup runs every thursday and the option "Verify after backup completes" is enabled and so the verify runs directly after the completed backup.

Usually the whole full backup job of our fileserver runs for 5 hours, this includes ~ 45 minutes of verify. Our problem is that every second week the whole job needs 9 hours but the backup job doesn't need more time instead it's the verify job which runs for about 4 hours. It is really strange that the job needs the one week the normal time and the other week it's faulty and the verify needs 4 hours longer.

For example:

Thursday, September 1th -> verify 45 minutes,

Thursday, August 25th -> verify 4 hours 45minutes,

Thursday, August 18th -> verify 1 hour,

Thursday, August 11th -> verify 5 hours.....

I examined the last three months and I can not believe that this is just a coincidence?!

Any ideas what could cause this issue? Is there any option or operation/functionalty that would explain this behaviour?

Thanks in advance.

Backup Exec 2010 R2 - Rev. 4164

[sbora]

Is this backup going to tapes? Check if you have any other job using the tape drive at the same time. Do you see any alert in backup exec for this full backup job when the verify takes 4+ hours?

[aos-stade]

Thanks for your reply.

No, it is a backup-to-disc job and there are no alerts in the job log.

[sbora]

Any errors related to the disk holding B2D in event viewer? When the verify is slow at that time any additional activity on the disk compared to the week when the verify is fast. 

Issue seems to be environment specific and not with Backup Exec so i will recommend to investigate the differences in environment/server on the working and problem day.

[aos-stade]

Thanks for your reply.

There are no suspicious errors in the event viewer. The backup and verify jobs run at the night and there are no other events or activities which run simultaneously.

I did some other research and came to the conclusion that restarting the backup exec services definitely has changed something. I stopped and restarted the backup exec services on every wednesday before the full backup on thursday during the last 2 weeks:

Verify on thursday, September 1th was fine and therefore the verify on thursday, September 8th must have had the verify issue up to my state of knowledge. But on wednesday, September 7th I stopped and started the backup exec services as said before. And what should I say, the verify ran without any issues! I validated this behaviour and repeated the procedure a second time on the following week and the job was fine, too.

Next week I will not restart the services and I think the verify issue will be back again.

I don't think it is just an environment specific issue. I think the backup exec services are involved, but I will see that next thursday.