0xe000fedf - A failure occurred reading an object. Shadow Copy Components.

[Lange_G]

Hi folks,

I have been battling with the 0xe000fedf - A failure occurred reading an object. error for some days now.

It occurs on Shadow Copy Components and the backup fails most of the time, though very occasionally it completes successfully with no error. Almost every time the error lists a different component as a corrupt file. For example:

WARNING: "Shadow Copy Components\System Service\Windows Management Instrumentation\WMI" is a corrupt file.

WARNING: "Shadow Copy Components\System Service\Event Logs\Event Logs" is a corrupt file.

WARNING: "Shadow Copy Components\System State\System Files\System Files" is a corrupt file.

I have tried the following:

Created a Backup Job with only Shadow Copy Components selected and used it for testing.

Disabled Trend Micro ServerProtect real-time scan during backups - the errors still occured.

Backup Job - Tried with Advanced Open File Option (MS Volume Shadow Copy Service) enabled and disabled - either way the errors still occured.

Enabled Debug Logging and tried to trace the error in the resulting logs by following the instructions here: http://www.symantec.com/business/support/index?page=content&id=TECH50883 - unfortunately the error changes each time and doesn't follow the format in this article.

My client is running SBS 2003 SP2 in a single server environment with Symantec Backup Exec 10d for Windows Servers version 10.1 Rev. 5629.

Any insights would be appreciated!

Cheers!

[pkh]

This is caused by the fact that these files are in used at the time of the backup.  If you are licenced for AOFO, then turn on AOFO.  This will take care of the problem.

[VJware]

Does NTBackup also skip the files ?

Also, do enable  " File system uses asynchronous I/O " if turned off using - http://www.symantec.com/docs/TECH42859

[Lange_G]

Hi VJware, thanks for your response. Here is what I've found.

Extracted from SERVER-beremote00.log

[0716] 02/19/12 23:10:25 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\x86_Microsoft.VC90.MFC_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_028bc148\' 'mfcm90u.dll' 'C:\WINDOWS\WinSxS\x86_Microsoft.VC90.MFC_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_028bc148\mfcm90u.dll'
[0716] 02/19/12 23:10:25 brUtil::OpenActiveObject Information: Active Object is a File C:\WINDOWS\WinSxS\x86_Microsoft.VC90.MFC_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_028bc148\mfcm90u.dll
[0716] 02/19/12 23:10:25 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.4148_x-ww_f0ccd4aa\' '' 'C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.4148_x-ww_f0ccd4aa\'
[0716] 02/19/12 23:10:25 brUtil::OpenActiveObject Information: Active Object is a Directory C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.4148_x-ww_f0ccd4aa\
[0716] 02/19/12 23:10:25 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.4148_x-ww_f0ccd4aa\' 'vcomp90.dll' 'C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.4148_x-ww_f0ccd4aa\vcomp90.dll'
[0716] 02/19/12 23:10:25 brUtil::OpenActiveObject Information: Active Object is a File C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.4148_x-ww_f0ccd4aa\vcomp90.dll
[0716] 02/19/12 23:10:25 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_4ddc769f\' '' 'C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_4ddc769f\'
[0716] 02/19/12 23:10:25 brUtil::OpenActiveObject Information: Active Object is a Directory C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_4ddc769f\
[0716] 02/19/12 23:10:25 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_4ddc769f\' 'vcomp90.dll' 'C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_4ddc769f\vcomp90.dll'
[0716] 02/19/12 23:10:25 brUtil::OpenActiveObject Information: Active Object is a File C:\WINDOWS\WinSxS\x86_Microsoft.VC90.OpenMP_1fc8b3b9a1e18e3b_9.0.30729.6161_x-ww_4ddc769f\vcomp90.dll
[0716] 02/19/12 23:10:25 Status FS_STREAM_CORRUPT (0xE000FE36) for Component System Files in SHADOW::ReadComponent
[0716] 02/19/12 23:10:25 Informational: Closing Component 'System Files'
[0716] 02/19/12 23:10:25 Informational: calling IVssBackupComponents::SetBackupSucceeded with status 'FS_READING_OBJECT_FAILURE (0xE000FEDF)' for Component 'System Files' in SHADOW::CloseComponent

Checked NTBackup logs for "vcomp90.dll" and it is not there. I then searched C:\Windows\WinSxS and found the two copies of vcomp90.dll, one dated 2009 and the other 2011. I scanned them both at Virus Total to be safe and neither was detected as infected. Of course I also ran a chkdsk which shows no problems.

I noticed error 0xE000FE36 occurs in the SERVER-beremote00.log just before 0xE000FEDF in this instance which may help in troubleshooting the error.

The strange thing is that after this SERVER-beremote00.log file was created, I had a successful backup which included these components - which indicates to me that there may not be any corruption? And, I've also had backup jobs completing the System State, but failing in System Service (WMI) as I mentioned in my first post. Any ideas why this would be?

Regarding running BEutility, it doesn't list any Media Servers nor Media Server Groups. Should I create a media server then apply the "File system uses asynchronous I/O" setting? Please note that I didn't create this backup solution, I have been called in to fix it. The client is backing up to USB HDD media using B2D Folders currently, and previously they were setup to as B2D Removable Folders. I have tested with both and the errors occur with both.

Thanks again for the assistance!

[VJware]

Yes, create the media server & apply "File system uses asynchronous I/O" setting...

[Lange_G]

I selected/created the media server and found that the "File system uses asynchronous I/O" setting was already applied. I deselected it, saved, went back in to Edit Configuration and ticked to apply the setting then saved again.

Running the a test job came up with the following error:

Backup- Shadow?Copy?Components Shadow?Copy?Components

WARNING: "Shadow?Copy?Components\System?State\SYSVOL\SYSVOL\016b65f1-7b14-40bb-8208568f5de455f5" is a corrupt file. This file cannot verify.

Verify- Shadow?Copy?Components Shadow?Copy?Components

WARNING: "016b65f1-7b14-40bb-8208568f5de455f5" is a corrupt file. This file cannot verify.

I'm not sure what would be a next step. I contacted the clients Symantec partner vendor, who also originally installed the software (and setup all backup jobs without the verify option selected). So far they tell me that they have never heard of these errors before and they suggested that the client upgrade - but they can't say whether an upgrade will fix these problems.

Thanks again!

[VJware]

Confirm if the BESA has appropriate permissions using - http://www.symantec.com/docs/TECH136148

This BE service account have should full permissions on the C:\

[Lange_G]

Hi VJware, I'm not at the clients site now, but I will through this remotely over the next few days. I will run BE_SupportTool.exe tomorrow and get back to you.

Thanks again!

[Lange_G]

Hi VJware, sorry for the delay in getting back to you, I've been flat out.

I'm back at the client's site today and I have followed the instructions in TECH136148 as you suggested. The results from Backup Exec Support Tool v1.008.111 confirm that BE Service and BESA are running with the right credentials (the domain admin account).

The support tool did bring up an Issue - Are Window VSS writers stable? Status: Error

11 out of 13 VSS Writers were listed as State = STABLE, with the following 2 exceptions:

SqlServerWriter = VSS_WS_WAITING_FOR_BACKUP_COMPLETE

BITS Writer = VSS_WS_WAITING_FOR_BACKUP_COMPLETE

All 13 VSS Writers were listed as Error = N0 (no exceptions)

I have followed the links provided in the results and confirmed that the client is running MS SBS 2003 Service Pack 2 Build 3790. I also looked into http://support.microsoft.com/default.aspx?kbid=940349 - VSS update rollup - as a potential fix, but unfortunately it doesn't list SBS 2003 in the "applies to" section, though it could work.

Thanks again for your help so far, any further suggestions would be appreciated.

[Lange_G]

Hi VJware, I followed some of your other posts and I will try re-registering VSS and rebooting the server once the staff have left the office. I will try the process here: http://support.microsoft.com/kb/940032

Cheers!

[Lange_G]

Hi VJware, I have tried everything, restarting VSS components, restarting the server, and the error still occurs 90% of the time. The client can't wait any longer so I've had to look into other backup solutions which I won't mention on this forum. If you have any last ideas before I move them away from Symantec products please let me know!

Thanks again!