System State backup freezes

[oskaremil]

Last three weeks the System State backup on one of my servers has failed. There was a power failure around the times it started to fail but I cannot confirm the exact date, everything else is working after the hard reboot but it seems like the VSS writer has been disrupted. Reboot does not help.

In Backup Exec, the job freezes on System State and stays there until I cancel or the job runs out of its time window. I've tried to create a test job with only System State for that server but the result is the same.

vssadmin list writer gives me this (while BE is freezing on System State):

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Writer name: 'SqlServerWriter'
   Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
   Writer Instance Id: {759f9ac3-c0e0-4b81-a4d0-28e92bdcfd5e}
   State: [1] Stable
   Last error: No error

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {d8f783ae-c5cd-485f-8d39-7cc6c9f55d28}
   State: [5] Waiting for completion
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {f026532b-710d-47c1-ae9b-45c823b42982}
   State: [5] Waiting for completion
   Last error: No error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {66fd0720-c1b0-46d9-9d94-74038f08fc57}
   State: [5] Waiting for completion
   Last error: No error

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {e717e99a-2e1f-423a-9e38-1b74ee643ceb}
   State: [5] Waiting for completion
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {41a34512-dfd1-407d-b199-f9875b0c84ea}
   State: [5] Waiting for completion
   Last error: No error

Writer name: 'BITS Writer'
   Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
   Writer Instance Id: {f9139f0b-8b7e-472c-a839-ef6a9e7c8a90}
   State: [1] Stable
   Last error: No error

Writer name: 'IIS Config Writer'
   Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
   Writer Instance Id: {6f254043-e347-49c2-bc3c-1a88831c1b88}
   State: [5] Waiting for completion
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {6c504fe1-d1b1-4d67-b6cf-a428f15d2e03}
   State: [1] Stable
   Last error: No error

After a server reboot vssadmin shows all writers as [1] Stable, Last error: No error.

My Backup Exec version is 2010 (13.2896). I am currently downloading 2010 R3 to see if that helps though I doubt that since everything worked fine until three weeks ago.

I am in need for tips on where to continue the error-tracking and get the backup up and running again. Thank you in advance.

The problem server is a Windows 2008 64-bit server. There are no error/warning messages in the Event Viewer that give me any clues but there is a suspicious Information message that appears about a minute after the System State backup starts

Source:VSS, Ran out of time while deleting files.

According to Microsoft this message could be ignored, but I don't know ?

[AmolB]

Have you tried running backup using  Wbadmin?

[Zuby]

upgrade wont resolve your issue

system writers are not responding so we need to fix it in order to run successful system state backup

update OS with latest service pack and patches

let me know if it helps!

[Simon_B_]

If it is indeed that case that a snapshot froze with the crash and now does not delete its files then you could try the manual way.

use vssadmin list shadows (after a clean reboot) to determine if there are any snapshots left over. If so, use vssadmin delete shadows to delete them (syntax here: http://technet.microsoft.com/de-de/library/cc788026(WS.10).aspx)

Be aware that this might also clear shadow copy snapshots which were created to backup data on file shares - so be sure you know what you are deleting ;)

Also as stated above after that try to run a backup of the system state using the windows backup utility. If this still fails you would have to log a case with MS as this then would mean you essentially have some errors within the MS VSS funtion itself which BE also uses to backup the system state.

[oskaremil]

I tried wbadmin to make a system state backup, that job completed with success. Out of curiosity I tried running a system state backup from Backup Exec but that still failed.

vssadmin list shadows gives me this list, it's the shadow copies created by my wbadmin job, however vssadmin delete shadows fails, this is the command prompt output:

  C:\Users\Administrator>vssadmin list shadows
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool

(C) Copyright 2001-2005 Microsoft Corp.

Contents of shadow copy set ID: {91030756-661f-4f4b-b69b-2abc60d2b6db}

   Contained 2 shadow copies at creation time: 11.05.2011 08:58:09

      Shadow Copy ID: {64409ec7-f541-4375-8076-011e5246acbd}

         Original Volume: (C:)\\?\Volume{9994e941-d5ba-11de-ba42-806e6f6e6963}\

         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1

         Originating Machine: PREMFTP01

         Service Machine: PREMFTP01

         Provider: 'Microsoft Software Shadow Copy provider 1.0'

         Type: ApplicationRollback

         Attributes: Persistent, No auto release, Differential, Auto recovered


      Shadow Copy ID: {6099319e-1966-48fe-9df2-b21843445cd2}

         Original Volume: (A:)\\?\Volume{407e8344-d5c5-11de-9747-0019996a27f0}\

         Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy3

         Originating Machine: PREMFTP01

         Service Machine: PREMFTP01

         Provider: 'Microsoft Software Shadow Copy provider 1.0'

         Type: ApplicationRollback

         Attributes: Persistent, No auto release, Differential, Auto recovered



C:\Users\Administrator>vssadmin delete shadows /for=c: /all

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool

(C) Copyright 2001-2005 Microsoft Corp.


No items found that satisfy the query.


C:\Users\Administrator> 

I don't think the error is in MS VSS function itself since a normal Windows Backup will complete without any problems.

[Simon_B_]

Are you backing up this machine via remote agent or is it a virtual machine which gets backed up through AVVI?

Also run vssadmin list providers. Are there any providers shown besides the default microsoft provider?

[oskaremil]

It is a physical computer backed up via Remote Agent.

The only VSS provider on the computer is Microsoft Software Shadow Copy provider 1.0

[oskaremil · Accepted solution]

Upgrading to BE 2010 R3 did the trick for me