Forum Discussion

mbuk2k's avatar
mbuk2k
Level 2
14 years ago

A failure occurred reading an object - System State Errors

We're getting the following error, please could someone advise if they've seen anything similar?

 

Job Completion Status

Job ended: 02 June 2011 at 09:30:54

Completed status: Failed

Final error: 0xe000fedf - A failure occurred reading an object.

Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-65247

Errors

Click an error below to locate it in the job log

Backup- \\SQL01.domain.com\System?StateV-79-57344-65247 - A failure occurred reading an object.

Exceptions

Click an exception below to locate it in the job log

Backup- \\SQL01.domain.com\System?StateWARNING: "\\SQL01.domain.com\System?State\System Files\System Files" is a corrupt file. This file cannot verify.

BackupV-79-57344-34108 - An unexpected error occurred when cleaning up snapshot volumes. Confirm that all snapped volumes are correctly resynchronized with the original volumes.

Verify- \\SQL01.domain.com\System?State \\SQL01.domain.com\System?StateWARNING: "System Files" is a corrupt file. This file cannot verify.

We’re getting a backup job fail (it seems to be every other job) and it’s to do with the system state backup of a SQL server. I’ve attached debug logs from the BE agent from that SQL box and it appears to point to different files/issues each time it fails. Our customer is running Backup Exec 2010 and the SQL server is Windows Server 2003 x64.

To whittle it down to the system state on the SQL box I created a separate backup job that just backs up the system state from that server to disk (original backup was to tape and to multiple servers).

 

[2256] 06/02/11 09:10:35    Adding C:\WINDOWS\system32\dllcache\dmview.ocx to the backup file list
[2256] 06/02/11 09:10:35    Adding C:\WINDOWS\system32\dllcache\dnsperf.dll to the backup file list
[2256] 06/02/11 09:10:35 Status 0x00000002 returned calling FindNextFile for \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy154\WINDOWS\system32\dllcache\dnsperf.dll in brUtil::GenerateFileList
[2256] 06/02/11 09:10:35 Component Set System Files has 1 files missing on the snapshot
[2256] 06/02/11 09:10:35 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\system32\CatRoot\{127D0A1D-4EF2-11D1-8608-00C04FC295EE}\' '' 'C:\WINDOWS\system32\CatRoot\{127D0A1D-4EF2-11D1-8608-00C04FC295EE}\'
[2256] 06/02/11 09:10:35   AD:Status E_SNAPSHOT_FILE_MISSING_ON_SNAPSHOT (0xE0009420) calling FS_OpenObj in SystemState::OpenObj:172
[2256] 06/02/11 09:10:35 SHADOW:: C:\WINDOWS\system32\dllcache\* /s - E_SNAPSHOT_FILE_MISSING_ON_SNAPSHOT
[2256] 06/02/11 09:10:35 brUtil::SetMetaDataStream - Metadata Stream Header (439600)
[2256] 06/02/11 09:10:35 brUtil::OpenActiveObject Information: Active Object is a Directory C:\
[2256] 06/02/11 09:10:35 brUtil::OpenActiveObject Error: 0xE00084AF Could not open the object  C:\WINDOWS\system32\CatRoot\{127D0A1D-4EF2-11D1-8608-00C04FC295EE}\
[2256] 06/02/11 09:10:35 Status FS_NOT_FOUND (0xE00084AF) for Component System Files in SHADOW::ReadComponent
[2256] 06/02/11 09:10:35   AD:Status FS_READING_OBJECT_FAILURE (0xE000FEDF) calling FS_ReadObj in SystemState::ReadObj:262
[2256] 06/02/11 09:10:35 Informational: Closing Component 'System Files'
[2256] 06/02/11 09:10:35 Informational: calling IVssBackupComponents::SetBackupSucceeded with status 'FS_READING_OBJECT_FAILURE (0xE000FEDF)' for Component 'System Files' in SHADOW::CloseComponent
[2256] 06/02/11 09:10:35 Status VSS_E_WRITER_INFRASTRUCTURE (0x80042318) returned from querying writer metadata status in SHADOW::GetWriterStatus
[2256] 06/02/11 09:10:35 Status VSS_E_BAD_STATE (0x80042301) returned getting writer status count in SHADOW::GetWriterStatus
[2256] 06/02/11 09:10:35 Status VSS_E_BAD_STATE (0x80042301) returned getting Component 'System Files' Writer status in SHADOW::CloseComponent
[2256] 06/02/11 09:10:35   AD:Status VSS_E_BAD_STATE (0x80042301) calling FS_CloseObj in SystemState::CloseObj:107
[2256] 06/02/11 09:10:35 brUtil::GetSpecialObjectStream() - Getting special object from (C:)
[2256] 06/02/11 09:10:35 brUtil::GetSpecialObjectStream() - End of special object
[2256] 06/02/11 09:10:35 TF xfer time = 6 seconds.
[2256] 06/02/11 09:10:35 WRITE:  tpreceive_fail_count = 15
[2256] 06/02/11 09:10:35 WRITE:  waiting_on_buffers_count = 5
[2256] 06/02/11 09:10:35 WRITE:  buffers_written_count = 868
[2256] 06/02/11 09:10:35 TF_CloseSet()
[2256] 06/02/11 09:10:35 FreeFormatEnv( cur_fmt=0 )
[2256] 06/02/11 09:10:35 Detach from \\SQL01.domain.com\System?State
[2256] 06/02/11 09:10:35 Detach from Shadow?Copy?Components
[2256] 06/02/11 09:10:35 brUtil::brUtil Destructor
[2256] 06/02/11 09:10:35 Detach from C:
[2256] 06/02/11 09:10:35 FS_FreeOSPathOrNameQueueInHand: Error. QueueCount( &fsh->in_use_name_q ) != 0
[2256] 06/02/11 09:10:35 TF_FreeDriveContext( 4D3D010 )
[2256] 06/02/11 09:10:35 TF_FreeTapeBuffers: from 10 to 0 buffers
[2256] 06/02/11 09:10:35 Backup Job Stop(0)  - Thu Jun 02 09:10:35 2011
[2256] 06/02/11 09:10:35 FreeFormatEnv( cur_fmt=0 )
[2256] 06/02/11 09:10:36 VSS BackupComplete called with success.
[2256] 06/02/11 09:10:36 VssSnapshotVolume::GetFailedWriterStatus() - waiting for IVssBackupComponents::GatherWriterMetadata to complete.
[2256] 06/02/11 09:10:36 VssSnapshotVolume::GetFailedWriterStatus() - IVssBackupComponents::GatherWriterStatus complete.
[2256] 06/02/11 09:10:36 VssSnapshotVolume::GetFailedWriterStatus() - Issuing Async QueryStatus check.
[2256] 06/02/11 09:10:36 Error (0x80042318) from VSS -  VSS_E_WRITER_INFRASTRUCTURE
[2256] 06/02/11 09:10:36  after calling function: IVssAsync::QueryStatus: hr2
[2256] 06/02/11 09:10:36 Attempting to delete VSS snapshot set: {2daf5d90-0f0d-45c5-b0dd-8e54e2fae61d}
[2256] 06/02/11 09:10:36 Error (0x800706bf) from VSS - Unknown error code
[2256] 06/02/11 09:10:36  after calling function: IVssBackupComponents::DeleteSnapshots
[2256] 06/02/11 09:10:36 Error deleting VSS snapshot set. Snapshot SetID: {2daf5d90-0f0d-45c5-b0dd-8e54e2fae61d}
[2256] 06/02/11 09:10:36 Number of snaps deleted: 0. Failed Snapshot ID: {00000000-0000-0000-0000-000000000000}
[2256] 06/02/11 09:10:36 ERROR: ndmpdSnapshotDelete: Error after call to delete snapshot set
[2256] 06/02/11 09:10:36 @@@@@@@MyCloseSocket called with sockfd = 1204(0x4b4)    retval = 0
[2256] 06/02/11 09:10:36 FreeFormatEnv( cur_fmt=0 )
[2256] 06/02/11 09:10:36 FreeFormatEnv( cur_fmt=0 )

and another failure:

[3128] 06/02/11 08:25:15 brUtil::OpenActiveObject Information: Active Object is a File C:\WINDOWS\system32\dllcache\xpsshhdr.dll
[3128] 06/02/11 08:25:15 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\system32\dllcache\' 'xpssvcs.dll' 'C:\WINDOWS\system32\dllcache\xpssvcs.dll'
[3128] 06/02/11 08:25:15 brUtil::OpenActiveObject Information: Active Object is a File C:\WINDOWS\system32\dllcache\xpssvcs.dll
[3128] 06/02/11 08:25:15 brUtil:: 0x00000001 Could not get the next object to read. Probably end of list.
[3128] 06/02/11 08:25:15 Informational: Closing Component 'System Files'
[3128] 06/02/11 08:25:15 Informational: calling IVssBackupComponents::SetBackupSucceeded with status 'SUCCESS (0x00000000)' for Component 'System Files' in SHADOW::CloseComponent
[3128] 06/02/11 08:25:15 brUtil::GetSpecialObjectStream() - Getting special object from (C:)
[3128] 06/02/11 08:25:15 brUtil::GetSpecialObjectStream() - End of special object
[3128] 06/02/11 08:25:15 TF xfer time = 87 seconds.
[3128] 06/02/11 08:25:15 WRITE:  tpreceive_fail_count = 94
[3128] 06/02/11 08:25:15 WRITE:  waiting_on_buffers_count = 84
[3128] 06/02/11 08:25:15 WRITE:  buffers_written_count = 20256
[3128] 06/02/11 08:25:15 TF_CloseSet()
[3128] 06/02/11 08:25:15 FreeFormatEnv( cur_fmt=0 )

and another one:

[4520] 06/02/11 09:30:38 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\setuppolicies\' '' 'C:\WINDOWS\WinSxS\setuppolicies\'
[4520] 06/02/11 09:30:38 brUtil::OpenActiveObject Information: Active Object is a Directory C:\WINDOWS\WinSxS\setuppolicies\
[4520] 06/02/11 09:30:38 brUtil::SetupDLE Succesfully set up to read the object 'C:' '\WINDOWS\WinSxS\setuppolicies\amd64_policy.1.0.Microsoft.Windows.GdiPlus_6595b64144ccf1df_x-ww_AE43B2CC\' '' 'C:\WINDOWS\WinSxS\setuppolicies\amd64_policy.1.0.Microsoft.Windows.GdiPlus_6595b64144ccf1df_x-ww_AE43B2CC\'
[4520] 06/02/11 09:30:38 brUtil::OpenActiveObject Information: Active Object is a Directory C:\WINDOWS\WinSxS\setuppolicies\amd64_policy.1.0.Microsoft.Windows.GdiPlus_6595b64144ccf1df_x-ww_AE43B2CC\
[4520] 06/02/11 09:30:38 brUtil::OpenActiveObject Error: 0xE00084AF Could not open the object  C:\WINDOWS\WinSxS\setuppolicies\amd64_policy.1.0.Microsoft.Windows.GdiPlus_6595b64144ccf1df_x-ww_AE43B2CC\
[4520] 06/02/11 09:30:38 Status FS_NOT_FOUND (0xE00084AF) for Component System Files in SHADOW::ReadComponent
[4520] 06/02/11 09:30:38   AD:Status FS_READING_OBJECT_FAILURE (0xE000FEDF) calling FS_ReadObj in SystemState::ReadObj:262
[4520] 06/02/11 09:30:38 Informational: Closing Component 'System Files'
[4520] 06/02/11 09:30:38 Informational: calling IVssBackupComponents::SetBackupSucceeded with status 'FS_READING_OBJECT_FAILURE (0xE000FEDF)' for Component 'System Files' in SHADOW::CloseComponent
[4520] 06/02/11 09:30:38 Status VSS_E_WRITER_INFRASTRUCTURE (0x80042318) returned from querying writer metadata status in SHADOW::GetWriterStatus
[4520] 06/02/11 09:30:38 Status VSS_E_BAD_STATE (0x80042301) returned getting writer status count in SHADOW::GetWriterStatus
[4520] 06/02/11 09:30:38 Status VSS_E_BAD_STATE (0x80042301) returned getting Component 'System Files' Writer status in SHADOW::CloseComponent
[4520] 06/02/11 09:30:38   AD:Status VSS_E_BAD_STATE (0x80042301) calling FS_CloseObj in SystemState::CloseObj:107
[4520] 06/02/11 09:30:38 brUtil::GetSpecialObjectStream() - Getting special object from (C:)
[4520] 06/02/11 09:30:38 brUtil::GetSpecialObjectStream() - End of special object
[4520] 06/02/11 09:30:38 TF xfer time = 62 seconds.
[4520] 06/02/11 09:30:38 WRITE:  tpreceive_fail_count = 112
[4520] 06/02/11 09:30:38 WRITE:  waiting_on_buffers_count = 102
[4520] 06/02/11 09:30:38 WRITE:  buffers_written_count = 14197
[4520] 06/02/11 09:30:38 TF_CloseSet()
[4520] 06/02/11 09:30:38 FreeFormatEnv( cur_fmt=0 )
[4520] 06/02/11 09:30:38 Detach from \\SQL01.domain.com\System?State
[4520] 06/02/11 09:30:38 Detach from Shadow?Copy?Components
[4520] 06/02/11 09:30:38 brUtil::brUtil Destructor
[4520] 06/02/11 09:30:38 Detach from C:
[4520] 06/02/11 09:30:38 FS_FreeOSPathOrNameQueueInHand: Error. QueueCount( &fsh->in_use_name_q ) != 0
[4520] 06/02/11 09:30:38 TF_FreeDriveContext( 4CE1C60 )
[4520] 06/02/11 09:30:38 TF_FreeTapeBuffers: from 10 to 0 buffers
[4520] 06/02/11 09:30:38 Backup Job Stop(0)  - Thu Jun 02 09:30:38 2011
[4520] 06/02/11 09:30:38 FreeFormatEnv( cur_fmt=0 )
 

Does anyone have any suggestions on how we can help resolve the issue?

Thank you

2010
Agents
Backing Up
Backup and Recovery
Backup Exec
Error messages
Managing Backup Devices
Troubleshooting
  • mbuk2k's avatar
    mbuk2k
    14 years ago

    Thanks again for the reply.

    I had seen that article but wanted to check other options before reinstalling SP2.

    It turns out that the issue was due to vReplicator killing the Microsoft VSS writer every five minutes. This meant that some jobs succeeded and some failed. Once this had been stopped the backups ran fine. We're going to take it up with the vendor to find an appropriate workaround.

    Thanks for your help.

4 Replies

Sort By
  • Dev_T's avatar
    Dev_T
    Level 6
    14 years ago

    Hi,

    V-79-57344-34108 - An unexpected error occurred when cleaning up snapshot volumes

    If running an ANTIVIRUS like MCAFEE on the affected server; try disabling that.

  • mbuk2k's avatar
    mbuk2k
    Level 2
    14 years ago

    Thank you for your response.

     

    No AV is running on this box it seems, any other ideas?

     

    Thanks again

  • Dev_T's avatar
    Dev_T
    Level 6
    14 years ago

    Hi,

    According to the Symantec article http://www.symantec.com/docs/TECH50883

     

    5. Another way to confirm that this file is corrupt is by running a backup of the System State with NT Backup. Note** NT Backup will not fail with any errors during the backup, instead the report from the backup will need to be reviewed.

    Follow these steps to use NT Backup:

    a. Click on Start -> All Programs -> Accessories -> System Tools -> Backup
    b. Uncheck "Run in wizard mode" and click Cancel
    c. Repeat step a. to open NT Backup again.
    d. Click on Tools -> Options -> Backup Log and select Detailed
    e. Click OK
    f. Click on the Backup tab and select the System State
    g. For Backup Destination select File and choose a location and file name.
    h. Click Start Backup
    i. Click Advanced and select Verify data after backup
    j. Click OK
    k. Click Start Backup
    l. When the backup finishes click on the Report button to see the report
    m. Save the report as a txt file.
    n. In the report search for the corrupt file - in this example "sfcfiles.dll" - notice that the file will not be listed in the report meaning that NT Backup did not even backup the file. Even though no errors were reported, this confirms that the file is corrupt.
     

    SOLUTION:

    To resolve this issue the following steps would help:

    1. If the Operating System was installed with a Service Pack - such as SP1 for Windows 2003 - they will need to re-apply the service pack and download all Windows updates.

    2. In the case of Windows 2003 R2 - run an in-place upgrade to R2 and apply updates.

    3. If running an ANTIVIRUS like MCAFEE on the affected server; try disabling that.

    4. If the customer does not want to re-install the service pack or apply R2 to their existing Operating System then speak with Microsoft to resolve the corrupt file.

    Once the file is successfully backed up using NT Backup then Backup Exec will be able to perform a backup without error

    Hope this helps...

     

    Please update...

  • mbuk2k's avatar
    mbuk2k
    Level 2
    14 years ago

    Thanks again for the reply.

    I had seen that article but wanted to check other options before reinstalling SP2.

    It turns out that the issue was due to vReplicator killing the Microsoft VSS writer every five minutes. This meant that some jobs succeeded and some failed. Once this had been stopped the backups ran fine. We're going to take it up with the vendor to find an appropriate workaround.

    Thanks for your help.