BE 2010 R3 and system state failure

[dpelligra]

Hi,

I have troubles backing up data with BE 2010 R3 on Windows 2008 R2

The failure occurs with System State : 0xe000fedd - A failure occurred accessing the object list

I have to select the whole system state in selection list and cannot select any subdir so I can't really see how i could troubleshot this ?

What I know is that BE is using VSS to backup systemstate and hat 12GB are backed up before the error occurs.
Also a file is ignored : System?State\System Files\System Files

System State contains the following subdirs on my selection list :

- AD
- ASR
- COM+ Class Registration Database
- Internet Information Services
- Performance Counter
- Registry
- System Files
- SYSVOL
- Task Scheduler
- VSS Expess Writer Store

Also notice BE 2010 R2 used to be installed, then ARCServe R16 i tried a few days, then my BE 2010 R3 in trouble I intend to keep.

I had VSS issues earlier with an old ARCServe VSS writer that was still in the system in spite of the complete uninstallation I made.

Any idea welcome to diagnose that :)

Thanks,

[ZeRoC00L]

Run vssadmin list providers right after a backup, and see what vss provider has an error state.

Also check this technote:

http://www.symantec.com/business/support/index?page=content&id=TECH60207

[AshwinApte]

Hi,

The remote Windows 2008 R2 server seems to be a domain controller, so one thing you can try is to run a backup without the AOFO (Advanced open File) option deselected.

Also make sure that you have completely removed any ArcServe agent installed on the remote Windows 2008 R2 domain controller.

Also check the event logs for any VSS related errors and if there are any errors there, please post them here.

Thanks

[CraigV]

...another thing to check is to see if your AV isn't perhaps blocking BE from accessing the registry. If so, put in an exclusion for the BE services on the DC in question.

THanks!

[dpelligra]

I check the VSSadmin list providers right after a backup and it only shows "Microsoft Shadow Copy provider" with no error. vssadmin list writers gives me a "request timed out" (i translated the french message to english myself) for one writer : ASR Writer

No error in event viewer.

For the record there's no remote server as BE 2010 R3 is installed on my DC.

AOFO was off, I tried putin it on with no success. I chose Microsoft VSS Service (windows 2003 and newer)

I made sure all non needed softwares are uninstalled, including ARCServe and BE 2010 R2.

I thought of the AV at first and uninstalled it a few days ago, still no success.

Thanks,

[CraigV]

...then your VSS writers are in error. You can fix that in 1 of 2 ways:

1. Restart the server.

2. Reregister the VSS *.dlls (no restart required):

http://www.symantec.com/business/support/index?page=content&id=TECH70486

[dpelligra]

I did as suggested, reregistering the DLLs and vssadmin list writers shows no more error.

I still get the same error though (it was not the case before my first post but I know backup only system state in real condition aka on a tape for the backup to be quick and to see the error come (or not) quicker)

Thanks,

[marcusolini]

Hi,  Can the specific System State component that is causing the failure be pinpointed?   It seems like the possible culprit is the System Files component.  Depending on which component is indentified, please attach the specific component’s metadata file or all of the metadata files to this post.   The component’s metadata file is located in Backup Exec logs directory.  For example: ‘c:\program files\symantec\backup exec\logs\{E8132975-6F93-4464-A53E-1050253AE220} - System Writer.xml’.  Thanks.

[dpelligra]

Hi,

I can't see how i could pinpoint the component failing since i can't deselect any component in my selection.

Find attached my metadata files.

Thanks,

EDIT : I just tried moving those files for BE to recreate them, with no success for my backup tho

[CraigV]

...if your media server is fully updated, and if you have pushed these updates out to your remote servers, then try running a backup of the System State to disk, and post back with the results.

Thanks!

[dpelligra]

I backed up on a local drive with the same issue at the end

Thanks

[marcusolini]

Thank you for the information thus far.  Unfortunately, it is still unclear at the moment as to what is causing the failure.  The current theory is that an unexpected snapshot exclusion is being reported, which would affect the System State backup and volume backup.  When able, please collect the Windows BackupRestore Registry Key and Backup Exec Engine debug logs from the affected domain controller for further review.  To confirm, the Backup Exec Media Server is installed on the Fr. Domain Controller (DNXSRV-AD1.dnxcorp.priv) and is backing up itself.  Thanks for your patience.

Collect the Windows BackupRestore Registry Key by capturing the following command line output:

C:\>REG QUERY "HKLM\SYSTEM\CurrentControlSet\Control\BackupRestore" /s

Collect the Backup Exec debug logs by first enabling Engine debug logging:

C:\>REG ADD " HKLM\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\Logging" /v "CreateDebugLog" /t REG_DWORD /d "1" /f

And then by re-running the failing System State and volume backup jobs

And finally by collecting the resulting Engine debug logs from the Backup Exec logs directory.  For example:

‘c:\program files\symantec\backup exec\logs\DNXSRV-AD1-bengine00.log’  and

‘c:\program files\symantec\backup exec\logs\DNXSRV-AD1-beremote00.log’.

Once complete, the Engine debug logging can be disabled:

C:\>REG ADD " HKLM\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\Logging" /v "CreateDebugLog" /t REG_DWORD /d "0" /f

[dpelligra]

Hi,

Here are the files you asked for

Thx,

[marcusolini]

The failure appears to be occurring due to an unexpected forward slash within a directory path.  Specifically, for the following file path reported by the System Files VSS Writer: “c:\program files/common\mili2service.exe”.

Please collect the Windows Services Key to determine if this is a 3rd party Windows Services being reported by the System Files VSS Writer.  Thank you for your efforts needed to collect all of this information.

Collect the Windows Services Registry Key by capturing the following command line output:

REG QUERY "HKLM\SYSTEM\CurrentControlSet\Services" /f "mili2service.exe" /s

[dpelligra]

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Emulex Management Interface Library v2
    ImagePath    REG_EXPAND_SZ    C:\Program Files/Common\MILI2Service.exe

1 match found

That path is wrong i just can't find the "common" dir.

Mili2Service.exe is found in C:\Program files\Emulex\Util\Common

Should I edit the BE key ?

Thanks

[marcusolini · Accepted solution]

Looks like you have a couple of choices on how to correct this.  (1) Modify the ImagePath by changing the file path from “/” to “\”  or (2) Modify the ImagePath by changing the entire file path to the correct location "C:\Program files\Emulex\Util\Common\mili2service.exe".

I would recommend option (1).   Keep in mind, that since the file path does not exist the System State informational backup exception may be present: “VSS Snapshot warning. File c:\program files\common\mili2service.exe is not present on the snapshot.”

Only consider option (2) if you are fully familiar with this Windows Service since this option will result in a different behavior for this Windows Service.

When able, please let me know the outcome.  Thanks again for all of your efforts and patience.

[dpelligra]

Hi :)

Editing the reg key worked just fine =)

Since i don't want any exception to show I put the right path and the backup is doing OK. Now I have to handle a tape pool to make it work perfectly, but that will be another thread later ;)

Thanks to everone and thanks to marcusolini ;)

[dpelligra]

Hi :)

Editing the reg key worked just fine =)

Since i don't want any exception to show I put the right path and the backup is doing OK. Now I have to handle a tape pool to make it work perfectly, but that will be another thread later ;)

Thanks to everone and thanks to marcusolini ;)

[marcusolini]

Great news!  And thanks to you!  :)