Snapshot provider error (0xE0008516): Resources specified for

[Jasonn]

BE Server Details:

BE Server OS: Windows Server 2008 Ent Sp2 x64

BE Version: Ver 13.0 Rev 2896 (64-bit)

Target Details:

Hyper-V cluster, errors occur on Guests on one of the cluster hosts only.

Host Server OS: Windows Server 2008 R2 Ent x64

Failed Guests:

Guest 1: MS Win 2K8 R2 Std x64 running MS DNS only

Guest 2: MS Win 2K8 R2 Std x64 running voip application

Guest 3: MS Win 2K8 R2 Std x64 running Exchange 2010

Same error occurs on all three guests

V-79-57344-34070 - AOFO: Initialization failure on: "VRTSRV::\\HV_CLUSTER.DOMAIN.LOCAL\Hyper-V?HA?VM\guest3". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).

Snapshot provider error (0xE0008516): Resources specified for snapshot do not have any valid data on them. Check if files were deleted or renamed.

Check the Windows Event Viewer for details.

Notes:

there are many additional systems both virtualised as part of the same cluster on a second node, on other hyper-v hosts and normal physical servers. It is only these 3 hosts that have the above error messages who all reside on the same host.

Anyone able to help resolve this?

[NeerThadarai]

Have you checked and tried this

http://www.symantec.com/docs/TECH128934.

Also it would be a good idea to install Remote Agent on those Hyper-V guest and try the backup to see if that runs successfull or does it also fails.

[Jasonn]

Vhdmount works fine.

running the job with Sgmon now.

Not this is all disk to tape. I did do a couple of disk to disk test a couple of weeks back with a vm guest on this host and that worked ok. it only seems to be with the tape job.

Will report back after I've got the sgmon log

[Jasonn]

These are some of the errors showing in the sgmon trace:

BENGINE:  [12/31/10 08:39:36] [0000]     [1364] 12/31/10 08:39:36 "Cluster" key does not appear to be present in the registry

BENGINE:  [12/31/10 08:39:36] [0000]     [1364] 12/31/10 08:39:36 Failed to open Microsoft cluster ()

BENGINE:  [12/31/10 08:39:36] [0000]     [1364] 12/31/10 08:39:36 VCS cluster keys do not appear to be present in the registry

BENGINE:  [12/31/10 08:39:36] [0000]     [1364] 12/31/10 08:39:36 Failed to open VCS cluster ()

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 Notes Agent:CLNFileSystem::ResolveDeviceName

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 ADC_ResolveDeviceName: pid = 4012, checking VRTSRV::\\HV_CLUSTER.DOMAIN.LOCAL\Hyper-V?HA?VM\guest3

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 ApplyRegExp(): Invalid input (VRTSRV::\\HV_CLUSTER.DOMAIN.LOCAL\Hyper-V?HA?VM\guest3). Parsing Failed.

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 GoodEvName(): Invalid EV device (VRTSRV::\\HV_CLUSTER.DOMAIN.LOCAL\Hyper-V?HA?VM\guest3).

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 NDMPAgentConnector::EndConnection(): Data Server = BACKUPSRV

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 NDMP_SSPI: reverted to user SYSTEM

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 NDMPAgentConnector:SetupConnection(): Data Server = HV_CLUSTER.DOMAIN.LOCAL

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 @@@@@@@MyCloseSocket called with sockfd = 1156(0x484) retval = 0

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 FreeFormatEnv( cur_fmt=0 )

BEREMOTE: [12/31/10 08:40:48] [0000]     [8348] 12/31/10 08:40:48 FreeFormatEnv( cur_fmt=0 )

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 Could not resolve the "bews-ndmp" or the "ndmp" service, error code: 10109, using port 10000

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 ndmpConnectEx: Querying the neighbour advertisement cache to discover information on 'HV_CLUSTER.DOMAIN.LOCAL' ...

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48 NRDS API - client connected.

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48         Connecting to BE Database.

BESERVER: [12/31/10 08:40:48] [0000]     [6548] 12/31/10 08:40:48 -1 Client requested key (1293745430).

BESERVER: [12/31/10 08:40:48] [0000]     [6548] 12/31/10 08:40:48 01 Server Configuration: Client added: 9

BESERVER: [12/31/10 08:40:48] [0000]     [6548] 12/31/10 08:40:48 -1 Client 'BACKUPSRV' connected('','DOMAIN\backupexecadmin'): 0x92b5020

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48         Successfully connected to BE Database.

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48         Reading agent database record for HV_CLUSTER.DOMAIN.LOCAL.

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48         Found agent record 9 for HV_CLUSTER.DOMAIN.LOCAL.

BESERVER: [12/31/10 08:40:48] [0000]     [6548] 12/31/10 08:40:48 01 Server Configuration: Client removed: 8

BESERVER: [12/31/10 08:40:48] [0000]     [6548] 12/31/10 08:40:48 -1 Client 'BACKUPSRV' Disconnected:0x92b5020

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48         Disconnected from BE Database.

BENETNS:  [12/31/10 08:40:48] [0000]     [6844] 12/31/10 08:40:48 NRDS API - client disconnected.

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 ndmpConnectEx : Control Connection information: A connection was established between end-points 10.2.1.49:62395 and 10.2.1.38:10000.

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 NDMP version 3 connection CONNECTED

BENGINE:  [12/31/10 08:40:48] [0000]     BECryptoInit: BECrypto non-FIPS mode successfully enabled.

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 IsLocalAgent:beclass::BEConvertServerNamePlus returned error: 87

BENGINE:  [12/31/10 08:40:48] [0000]     [1364] 12/31/10 08:40:48 IsLocalAgent:beclass::BEConvertServerNamePlus returned error: 87

BENGINE:  [12/31/10 08:40:51] [0000]     [1304] 12/31/10 08:40:51 Updating status for: 'Backup 00010-Weekly Full-Daily Diff-Monthly Full-Daily Diff' (0x0 0xB)

BENGINE:  [12/31/10 08:40:51] [0000]     [1304] 12/31/10 08:40:51 Status for: 'Backup 00010-Weekly Full-Daily Diff-Monthly Full-Daily Diff' updated

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 NDMPAgentConnector::EndConnection(): Data Server = HV_CLUSTER.DOMAIN.LOCAL

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 NDMPAgentConnector:SetupConnection(): Data Server = guest3.BRS.LOCAL

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 Could not resolve the "bews-ndmp" or the "ndmp" service, error code: 10109, using port 10000

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 ndmpConnectEx: Querying the neighbour advertisement cache to discover information on 'guest3.BRS.LOCAL' ...

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52 NRDS API - client connected.

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52         Connecting to BE Database.

BESERVER: [12/31/10 08:40:52] [0000]     [6548] 12/31/10 08:40:52 -1 Client requested key (1293745434).

BESERVER: [12/31/10 08:40:52] [0000]     [6548] 12/31/10 08:40:52 01 Server Configuration: Client added: 9

BESERVER: [12/31/10 08:40:52] [0000]     [6548] 12/31/10 08:40:52 -1 Client 'BACKUPSRV' connected('','BRS\backupexecadmin'): 0x4e3bca0

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52         Successfully connected to BE Database.

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52         Reading agent database record for guest3.BRS.LOCAL.

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52         Found agent record 2 for guest3.BRS.LOCAL.

BESERVER: [12/31/10 08:40:52] [0000]     [6548] 12/31/10 08:40:52 01 Server Configuration: Client removed: 8

BESERVER: [12/31/10 08:40:52] [0000]     [6548] 12/31/10 08:40:52 -1 Client 'BACKUPSRV' Disconnected:0x4e3bca0

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52         Disconnected from BE Database.

BENETNS:  [12/31/10 08:40:52] [0000]     [6844] 12/31/10 08:40:52 NRDS API - client disconnected.

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 ndmpEstablishConnectionUsingNoSpecificAdapter: Could not connect to remote address FE80::1558:FB2F:97EA:F370 and port 10000 Errorno : 10022

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 ndmpConnectEx : Control Connection information: A connection was established between end-points 10.2.1.49:62397 and 10.2.0.24:10000.

BENGINE:  [12/31/10 08:40:52] [0000]     [1364] 12/31/10 08:40:52 NDMP version 3 connection CONNECTED

BENGINE:  [12/31/10 08:40:52] [0000]     BECryptoInit: BECrypto non-FIPS mode successfully enabled.

BENGINE:  [12/31/10 08:40:53] [0000]     [1364] 12/31/10 08:40:52 IsLocalAgent:beclass::BEConvertServerNamePlus returned error: 87

BENGINE:  [12/31/10 08:40:53] [0000]     [1364] 12/31/10 08:40:52 IsLocalAgent:beclass::BEConvertServerNamePlus returned error: 87

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 Failed to Load AllowExcludes key, We will update DR file evenif there are userExcludes. Error: 5

BEREMOTE: [12/31/10 08:48:54] [0000]     [6528] 12/31/10 08:48:54 FreeFormatEnv( cur_fmt=0 )

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 Enter NDMPSnapHostBackupEngine::SnapSingleVolume

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 NDMPHostBackupEngine::CollectMetaDataForOneVirtualMachineBeforeBackup: entered

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 NDMPHostBackupEngine::ConnectToVirtualMachine: entered PDI Operation Type 1 PDIDataTransferPhase 1

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 NDMPHostBackupEngine::ConnectToVirtualMachine: VM friendly name 'guest3' guest computer name ''

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 NDMPHostBackupEngine::ConnectToVirtualMachine: this VM does not exist

BENGINE:  [12/31/10 08:48:54] [0000]     [1364] 12/31/10 08:48:54 NDMPHostBackupEngine::ConnectToVirtualMachine: returning 0x0

[Jasonn]

Been working through this today, some of the warnings im getting further down are similar to:

Backup- VRTSRV::\\HV.DOMAIN.LOCAL\Hyper-V?Virtual?Machine\GuestV-79-57344-38726 - Backup Exec failed to connect to virtual machine 'Guest' and was unable to collect the necessary metadata to restore individual application items. You cannot perform GRT-enabled restores of application data from this backup.

---

Backup- VRTSRV::\\HV.DOMAIN.LOCAL\Hyper-V?Virtual?Machine\GuestV-79-57344-38730 - Backup Exec failed to authenticate with virtual machine 'Guest' and was unable to collect the necessary metadata to restore individual application items. You cannot perform GRT-enabled restores of application data from this backup.

---

I think the errors are being caused in the AOFO because of these authentication issues potentially. I've fixed the auth issues and done a single vm backup with no issues. Tonight will be the true test when the entire backup job runs.