Remote Agent terminated unexpectedly (Beremote.exe crashes - faulting module msvcr71.dll)

[David_Tobin]

For the last week, backups haven't worked on a client's SBS 2003 SP Server.  During this time, we've observed frequent " Backup Exec Remote Agent for Windows Systems service terminated unexpectedly" errors in the event log, with these often happening soon after the server is rebooted (or after the service is restarted).

Investigating further in the Application Event log, I found this error shown:

(Event ID 1000) ”Faulting application beremote.exe, version 11.0.6235.14, faulting module msvcr71.dll, version 7.10.3052.4, fault address 0x00002eee.”

This server was working fine with Backup Exec 11d ver 6235 for over a year, and "suddenly" stopped last weekend.  I'm not aware of ANY patches or updates that were installed on the server in the leadup to this event that might have broken Backup Exec.

Since then (but not before this), the server has been randomly rebooting with no indication of the reason (we see the server reboot with Event ID 6008 "The previous system shutdown at 4:04:33 PM on 21/11/2008 was unexpected" logged in Event Viewer).

The service seems to be crashing even if a backup job isn't active, and the server reboots are also somewhat random.  If I end all Backup Exec Services on the server, then the reboots don't appear to be happening.

We are only using Backup Exec to save the server's "C drive" and also the System State.  We're not backing up any remote computers, nor are we using any agents for Exchange/SQL/Lotus/etc...  It's a very "simple" backup scenario that's suddenly become very complicated for us!

I've done a lot of research in this forum, and also on Google, and I've seen many users reporting this problem, but haven't seen anything solutions presented to them that have been helpful in our situation.  Since encountering the problem, I upgraded from release 6235 (inc SP1 and hotfixes 23, 24) to release 7170 (inc SP3 and hotfixes 47, 48, 51, 46, 53 and 58 which were applied via LiveUpdate).  Even on the 7170 release, I'm still see the symptoms I've discussed.

Hoping that somebody out there has some answers!

Thanks...

[David_Tobin]

I made a successful backup last night (the first in over a week!).  I'm not sure which thing I did solved it, but to help others who might have similar problems, here's a list of the things that I did:

* Upgraded firmware for our tape drive, an HP Ultrium-1 SCSI, to the current version (upgraded from P53D to P61D).  Still had the same problem after this.* Tested the tape drive using HP Storage Works Library & Tape Tools.  All tests passed OK.

* Disabled all Backup Exec services on the server.  Observed a stable system (no random reboots) for 24 hours.  Restarted Backup Exec services and the server rebooted within 30 minutes!  Disabled Backup Exec services again, and ran several successful backups with NTBackup.  Decided that this proved that the problem was either with Backup Exec or with the Symantec tape driver used by Backup Exec.

* Used Backup Exec's TapeInst.exe program to remove the Symantec driver for our tape drive.

* Observed Event ID 15 in the Event Viewer "RSM cannot manage library Tape0. The database is corrupt" some time later.  Fixed that error with these steps - thanks to info from website of Nick Whittome "The Naked MVP" :

1. Stop the Removable Storage Service, if started.
2. Stop the Backup Exec Services.
3. Go into C:\Windows\System32\ntmsdata folder, backup the contents of this folder.
4. Delete the files in this folder.
5. Start the Removable Storage Service and new files will be created.
6. Start the Backup Exec Services.

* I also read responses from users on Nick Whittome's blog that the RSM Service isn't necessary for Backup Exec and that disabling it can help with the "Database is corrupt" error above, so I chose to disable that service.

* Re-enabled all the Backup Exec services again and ran my backup!

Of course, after 1 successful backup, I'm not 100% ready to call the problem "solved" - I'll keep watch over the server in the coming days and report back if I have any more problems.

This problem certainly caused me a lot of trouble over the last week, and I can only hope that others out there might benefit from my solution.

[David_Tobin · Accepted solution]

I've had stable backups on that server for several weeks now, and have also identified the source of the random reboots: a faulty power supply.

I suspect that what happened to cause our initial problems with Backup Exec is that it was running a backup job during one of the initial random reboots we experienced, and that caused corruption to the RSM database (which I described in my previous post).  Since repairing that database, I've been running Backups just fine (but I've also now got the RSM service disabled on our server since it doesn't appear to be necessary for Backup Exec).