Backup Exec 10d (5629) SP4 - Remote agent fails with event ID 4096

[DFY]

Hello Symantec support forums,

I have a recurring issue on one of my servers, where the remote agent crashes during a backup operation. The details of my configuration are as follows:

Media Server runs on Windows Server 2003 R2 SP2, Backup Exec is version 10d (5629) SP4.
Protected server runs on Windows Server 2003 R2 SP2, runs MS SQL 2005, Backup Exec remote agent is also updated to 10d SP4.

The backup job of the resources on the protected server fails with the following error codes:

0x00084f8 - The network connection to the Backup Exec Remote Agent has been lost. Check for network errors.
Final error category: Resource Errors.

When looking in the windows event viewer (application log) however, I find a different cause for losing connection to the remote agent:

Event ID: 4096
Source: VsJITDebugger
Description: An unhandled 32 exception occured in beremote.exe (5092). Just-In-Time debugging this exception failed with the following error: Debugger could not be started because no users is logged on.

And in the event viewer system log at the same time:

Event ID: 7031
Source: Service Control Manager
Description: The Backup Exec Remote Agent for Windows Servers service terminated unexpectedly. It has done this (1) times. The following corrective action will be taken in 60000 miliseconds: Restart the service.

These errors occur regardless of whether I do backups of shared folder resources only or if I also include the SQL databases on the server. I have tried removing and re-installing the remote agent, but the problem persists.

I have searched these forums and online for anyone else having this problem and all I could come up with was this thread from 2007 which had almost the same error description but no replies or suggestions on how to fix it:

http://www.symantec.com/connect/forums/v-79-57344-65072-0xe000fe30-communications-failure-has-occurr...

Any help would be greatly appreciated,

/DFY

[DFY]

Nobody has seen this error before?

/DFY

[Gilly-Bhoy]

Just run a backup of a couple of folders on the system drive of the protected server?
When the agent crashes, is there ANY data backed up at all?

Regards,
Gilly

[DFY]

The backup jobs seems to fail when backing up the SQL server, and the job fails with a 0 byte completion.

Last night I got a little closer to an actual error. The Just-In-Time debug errors weren't very helpful so I went back to debugging with DrWatson and now get this entry in the event viewer when trying to perform a backup:

Event ID: 4097
Source: DrWatson
Description: The application, C:\Program Files\VERITAS\Backup Exec\RANT\beremote.exe, generated an application error. The error occurred on [DATE/TIME]. The exception generated was c0000005 at address 01F22353 (bedssql2!GetTable)

Any ideas on where to go with the troubleshooting on this one?

/DFY

[Gilly-Bhoy]

Please see this note on it:
http://seer.entsupport.symantec.com/docs/243346.htm
Let us know how it goes...

Gilly

[DFY]

I think the problem is solved.

After re-installing the agent and still getting the same errors I took a look at the backup job settings. The backup job was doing the SQL server first and then ran through a list of network shares on the server. I tried moving the SQL server to the end of the selections and now the job would run until it hit the SQL server and then failed.

I tried splitting the job into two seperate jobs, one doing the SQL server and one doing the network shares. The job doing the SQL server would fail and the one doing the shares would complete successfully. Then I disabled AOFO on the SQL server job and voila, the job completed successfully.

So it seems it was using AOFO on a MSSQL 2005 server that caused the backup exec remote agent to crash.

/DFY