Hyper-V Agent backups fail with Snapshot provider error (0xE000FED1)

[hgcs]

Hi!

There are two Windows Server 2008 R2 Server virtual machine running on a Windows Server 2008 R2 with Hyper-V role server.

I backup them separately every day with Hyper-V Agent without error. A few days ago one of the backup job fails:

Backup- xx.xx.local V-79-57344-65233 - AOFO: Initialization failure on: "VRTSRV::\\xx.xx.local\Hyper-V?Virtual?Machine\xx1". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).

Snapshot provider error (0xE000FED1): A failure occurred querying the Writer status.

Check the Windows Event Viewer for details.

Writer Name: Microsoft Hyper-V, Writer ID: {66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}, Last error: The VSS Writer failed, but the operation can be retried (0x800423f3), State: Waiting for backup complete notification (5).

In the Hyper-V role servers event log: EventID: 34113, Source: Backup Exec.

On the Hyper-V role server for all writer's: State: [1] Stable, Last error: No error

On the backup failed virtual machine has writers error: 

Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {72db0e57-461b-4ac8-a8f9-ba4706d76edf}
State: [5] Waiting for completion
Last error: No error

Writer name: 'SqlServerWriter'
Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance Id: {62dc8328-c1b8-4130-a227-d1fba8599500}
State: [8] Failed
Last error: Inconsistent shadow copy

Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {af14b31f-0326-4f0b-978e-dd70d2ab58f2}
State: [5] Waiting for completion
Last error: No error

Restarting the virtual machine and the Hyper-V role server clear the writers errors but the backup job again fails.

I have already reinstalled the sql server writer.

I have checked this:

http://www.symantec.com/business/support/index?page=content&id=TECH143771&actp=search&viewlocale=en_US&searchid=1328520790719

I think is not a disk signature problem because the backup job ws succes for a few month.

What can I do?

[Striker_303]

Are you able to backup this this virtual machine(xx1) using be remote agent.

Try this machine backup by selecting it under windows system and check what error you get.

[hgcs]

Do you mean I should define a new backup job with Hyper-V agent for the whole machine and run?

I have another daily job with SQL Server Agent to backup up databases to tape. It works.

[Striker_303]

Try to backup this whole machine (VM), using be remote agent under domains

like a normal physical machine.

This job is to isolate,

If you have some VSS related issue ....this backup would also fail  with that error.

If its some disk signature issue using Hyper V agent.......this backup should work fine.

[pkh]

You might also want to try re-installing the RAWS in the VM.

[hgcs]

I have tried the next 2 jobs now:

1. Domains/domain name/Virtual_machine name/C:\a_file_folder and a MS SQL database. The job completed successfuly.

2. Domains/domain name/Virtual_machine name/C:\a_file_folder, system state and a MS SQL database. The job completed successfuly . 

What is next?

[Striker_303]

If normal machine backup is working properly,  then this might be a disk signature issue

http://support.microsoft.com/kb/975695

[hgcs]

This virtual machine has 3 virtual disks. Which one has the disk signature changed? Perhaps all? What might have caused?

[hgcs]

Here is the link which I used to check the host machine disk signature and the virtual machines disk signature:

http://blogs.technet.com/b/markrussinovich/archive/2011/11/08/3463572.aspx

I saw that they are different!

So it is not a disk signature issue!

What is next?

[hgcs]

I have reinstalled th RAWS in the VM and restarte the VM. The job is running with the same failure.