Backup Exec 2010 R3 Exchange mailbox restore issues

[Garen15]

i am currently testing Backup Exec 2010R3, i have a weird issue that i can't figure it out. 
i am trying to restore a "Sent items" folder for a user in Exchange 2007 SP3  (VMware hosted)

>>>>here is the error message in the backup Log: 
status: Failed 
Final error: 0xe000ff12 - A communications failure has occurred with an Exchange Store resource. 
Final error category: Resource Errors 

Restore- \\MAIL.DOMAIN.local\Microsoft Information Store\First Storage Group V-79-57344-65298 - The Exchange Store service is not responding. Backup set canceled. 

RestoreV-79-57344-65072 - Connection to the restore target has been lost. Operation canceled. 

A selection on device \\MAIL.DOMAIN.local\Microsoft Information Store\First Storage Group was skipped because of previous errors with the job. 

>>>On the backup exec server i am seeing a few errors in the server's event viewer 
Process beremote.exe requested memory dump but the maximum number of memory dumps has already been reached. 

Faulting application beremote.exe, version 13.0.5204.0, time stamp 0x4d976a80, faulting module MSVCP80.dll, version 8.0.50727.4053, time stamp 0x4a5938bd, exception code 0xc0000005, fault offset 0x000000000002bb72, process id 0xec0, application start time 0x01cc279eae07ec14. 

Backup Exec Alert: Job Failed 
(Server: "BEServer") (Job: "USER Sent Items Restore") Jacquie Sent Items Restore -- The job failed with the following error: A communications failure has occurred with an Exchange Store resource. 

>>>on the Exchange server i see a different error message 
Faulting application name: Remsrv.exe, version: 13.0.5204.0, time stamp: 0x4d94c9bb 
Faulting module name: Remsrv.exe, version: 13.0.5204.0, time stamp: 0x4d94c9bb 
Exception code: 0xc0000005 
Fault offset: 0x000078b4 
Faulting process id: 0x13bc 
Faulting application start time: 0x01cc279b7575be63 
Faulting application path: C:\Program Files\SYMANTEC\BACKUP EXEC\RAWS\x86\Remsrv.exe 
Faulting module path: C:\Program Files\SYMANTEC\BACKUP EXEC\RAWS\x86\Remsrv.exe 
Report Id: 3c320792-9391-11e0-8037-000c2938b4d1 

Fault bucket , type 0 
Event Name: APPCRASH 
Response: Not available 
Cab Id: 0 

Problem signature: 
P1: Remsrv.exe 
P2: 13.0.5204.0 
P3: 4d94c9bb 
P4: Remsrv.exe 
P5: 13.0.5204.0 
P6: 4d94c9bb 
P7: c0000005 
P8: 000078b4 
P9: 
P10: 

Attached files: 
C:\Windows\Temp\WERA5BE.tmp.appcompat.txt 
C:\Windows\Temp\WERAB7B.tmp.WERInternalMetadata.xml 
C:\Windows\Temp\WERAB9C.tmp.hdmp 
C:\Windows\Temp\WERB2C1.tmp.mdmp 

These files may be available here: 
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_Remsrv.exe_4fbe22466df902f54e377af7745b9da382aeaa_cab_1420b3c7

Analysis symbol: 
Rechecking for solution: 0 
Report Id: 3c320792-9391-11e0-8037-000c2938b4d1 
Report Status: 4 

>>>>can anyone make any sense of this, i have been trying to fix this issue for the past 2 Days. 

the BE server is fully updated, there are no more updates showing in the LiveUpdate

the updated client agents have been pushed out to all server

let me know if you need more into to solve this issue.

[Garen15]

Anyone???

[CraigV]

Hi Garen,

OK, I assume that the RAWS agent is successfully installed on the remote server and is in a running state? Along with this, it should be publishing back to your media server.

What happens with the errors there is that when beremote.exe crashes, everything else in the job crashes = failure!

I'd start with a restart of the media server and try the restore again. If you're doing the restore directly from tape, it stages temporarily to C:\Temp so make sure you have enough disk space there, or move the temporary staging location to a drive with free space equivalent to the size of your Information Store.

If the restart doesn't sort it out, uninstall the RAWS agent, and then do a push-install again. The last thing to try for now would be to uninstall the Exchange agent from the media server, and then reinstall it.

Also make sure you have the Exchange 2007 management tools installed on the server as this is a requirement.

Thanks!

[Garen15]

Hi Craig,

thank you for your reply, the Exchange agent is definitely installed

i tried uninstalling and reinstalling it with no change to the situation

the Exchange management studio is installed on the Media server

I also tried uninstall and reinstalling it

i am restoring from a disk based system

the restore definitely starting and i can see data getting restored, but it always stops at this point

Byte count: 34,476,727 bytes Job rate: 22.00 MB/Min (Byte count of all backup sets divided by Elapsed time for all backup sets)

what do you mean by

The last thing to try for now would be to uninstall the Exchange agent from the media server, and then reinstall it

i don't have an exchange agent installed on the media server

can you give me more details on that point please?

is there some other troubleshooting steps i can try?

any ideas will be appreciated

[CraigV]

...the agent is activated when you install the license. That allows you to see Exchange instances on any other server with a remote agent.

[Garen15]

Hi Craig

yes i can see the Exchange instances in backup exec and the exchange backup is working fine

i have attached a screen shot

[ZeRoC00L]

The BEREMOTE.EXE process stops on the Exchange server, therefore the communication between the backup exec media server and the exchange servers stops.

First try to see (in the evenlog) if more errors occur at the same time the BEREMOTE process terminates.
Also check the performance of the server during a restore, if it is running out of recources (CPU, memory, disk).

Otherwise try to remove the BE agent on the exchange server, reboot and reinstall the BE agent.
 

[Garen15]

Hi ZeRoC00L,

i did another test right now, the BEREMOTE.EXE on the Backup Exec server is failing not the exchange server one.

the restore is starting, once it gets to 34,476,727 bytes it fails

the MB/Min starts dropping from 35MB to 10MB slowly then it fails.

i have attached the error messages.

it almost seems like it's chocking on a certain item in the restore :(

is there a way i can see each item being restored in a log file?

that way i can see which item it's failing on and exclude that item when restoring. what do you think?

[Garen15]

ok i figured out how to run it in debug mode

i did BEREMOTE.exe -debug in the services

i see an issue, not sure if it's the cause

the backup is on drive F and it's using Drive F for staging

how can i change it to use drive D:

----------------------

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  TempPath=F:\TEMP\Backup Exec\IMG24\vdb_2011_04_10_1146_30\tmp.edb

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  LogFilePath=F:\TEMP\Backup Exec\IMG24\vdb_2011_04_10_1146_30\

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  SystemPath=F:\TEMP\Backup Exec\IMG24\vdb_2011_04_10_1146_30\

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  altDbRecoveryPath=F:\TEMP\Backup Exec\IMG24\vdb_2011_04_10_1146_30\

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  BaseName=E00

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  0x24=2(0x2)

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  paramRecovery=0, on

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  paramDatabasePageSize=8192

[4368] 06/20/11 13:59:32 [fsys\jet]           - PARAM:  JET_paramOutstandingIOMax=1024

[4368] 06/20/11 13:59:32 [fsys\jet]           - STARTING RESTORE MAP: 

SOURCE PATH:  F:\IMG000523

TARGET PATH:  F:\TEMP\Backup Exec\IMG24\vdb_2011_04_10_1146_30

------------------------------

i can see the restore starting here

----------------

[4368] 06/20/11 13:59:34 [fsys\mb2]           - <<<< BACKUP >>>> Opening folder: Top of Information Store

[4368] 06/20/11 13:59:34 [fsys\mb2]           - <<<< BACKUP >>>> Opening folder: Top of Information Store

[4368] 06/20/11 13:59:34 [fsys\mb2]           - <<<< BACKUP >>>> Opening folder: Sent Items

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ----- ATTACHMENT TABLE FROM DB -----

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - [0] Attachement - Render position = 0x0

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ----- END ATTACHMENT TABLE FROM DB -----

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - No MIME for STREAM.

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ^^^^ ATTACH: (1-24) image003.jpg - Found data in N3701

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ----- ATTACHMENT TABLE FROM DB -----

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - [0] Attachement - Render position = 0x0

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ----- END ATTACHMENT TABLE FROM DB -----

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - No MIME for STREAM.

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ^^^^ ATTACH: (1-24) image002.jpg - Found data in N3701

[4368] 06/20/11 13:59:34 [fsys\mb2\edbprov]   - ----- ATTACHMENT TABLE FROM DB -----

-------------------------------------

and then way down at the end of the list i get this

----------------

[4368] 06/20/11 13:59:58 [fsys\mb2\edbprov]   - ----- ATTACHMENT TABLE FROM DB -----

[4368] 06/20/11 13:59:58 [fsys\mb2\edbprov]   - [0] Attachement - Render position = 0x0

[4368] 06/20/11 13:59:58 [fsys\mb2\edbprov]   - ----- END ATTACHMENT TABLE FROM DB -----

[4368] 06/20/11 13:59:58 [fsys\mb2\edbprov]   - No MIME for STREAM.

[4368] 06/20/11 13:59:58 [fsys\mb2\edbprov]   - ^^^^ ATTACH: (1-24) Glacie~1.jpg - Found data in N3701

[1012] 06/20/11 14:00:02 

For Virt VFF id (1012-000003F4), Total Bytes Read=00000000:025D2000, Written=00000000:00000000

[4000] 06/20/11 14:00:02 

For Virt VFF id (4000-00000FA0), Total Bytes Read=00000000:025E1000, Written=00000000:00004000

------------------

[CraigV]

Hi Garen,

If it's on disk it shouldn't need staging to my knowledge, but if you want to change it, you can do it as follows:

Tools --> Options --> Exchange

Look for the entry: C:\Temp, and change that to D:\Folder_Name . For future Exchange restores it will stage there first before restoring...

[Garen15]

Hi Craig,

i just checked the tools-->options-->Exchange

it's pointing to C:\Temp which is fine for now

but the weird thing is, this is showing in the log

TARGET PATH:  F:\TEMP\Backup Exec\IMG24\vdb_2011_04_10_1146_30

does that make sense to you? not sure why exactly

actually i was able to restore the sent items, what i ended up doing was find out at which point it failed

by going into outlook and finding the last restored email

then i went into Backup Exec's exchange restore selection list and selected the emails that are coming after the failed email and it went through and the restore completed fine :SSSSS

any ideas of what might be going on?

it was always failing at the same byte count

[CraigV]

...it's already on disk, and I assume you back up the the F:\ drive? If so, that is where it stages too...