A failure occured accessing the SnapShot handle.

[Mgamerz]

At my organization we have about 50 computers. We used to have Windows 7 32 bit but we have been upgrading them into Windows 7 64 bit. These machines were previously backed up on a regular basis. It seems that after I switch them to 64 bit, the System State throws this message:

(Server: "BACKUPSERVER") (Job: "Bigbird Full") Bigbird Full -- The job failed with the following error: A failure occurred accessing the SnapShot handle.

This happens on all of the machines that have either been upgraded or replaced with new computers. I have read all the technotes to turn off AOFO, but it doesn't make any difference. I am not sure what is causing this. I'm not too keen on giving Symantec a call given previous phone calls I've given them. I've also deleted and recreated the job, but it made no difference.

All the backups complete successfully, but we get sent FAILED messages and my boss... really doesn't like those.

Also, I don't understand AOFO. Does it use a VSS with AOFO on Windows 7? Do you leave it off and it uses VSS anyway? They don't describe it very clearly.

Thanks,

Michael

[SuperBrain]

AOFO only needs to be specifically selected for XP... for Windows 7 it will use VSS regardless.. 

Do you have any virtual machines being backed up in the same job?

When you say "All the backups complete successfully, but we get sent FAILED messages" do you mean the backups complete with exceptions?

[Mgamerz]

The description in backup exec and on the website(s) are... somewhat conflicting. I have AOFO on all our machines. The machines they are backing up are just standard Windows 7 enterprise workstations. No VMs, no Windows servers, just Windows installations.

I read somewhere that the failed snapshot message throws a 'FAILED' message even if the backup is successful (all the data is present on tape). I'll set half of them back to no AOFO and see if that makes a difference again.

[marcusolini]

Hi.  When able, please attach a corresponding Backup Exec Job Log that contains the failure to this post for review.  Thank you.

[pkh]

After you upgrade the machines to 64-bits, did you push out the remote agents again to these upgraded machines?

You need to turn on AOFO in order to backup opened files.  Otherwise, opened files would be skipped and you would have an error message.

[Mgamerz]

(To the other post) Yes, I have pushed out the agent. The backup goes fine until it accesses the System State. I'm getting two conflicting statements in this thread. Do I need AOFO installed and checked to backup open files on Windows 7 or do I not (Other person says it uses VSS anyways)?

Here is the job log (I changed the computer names since I don't think we're allowed to share those outside of our network)

This job is for a Windows 7 machine that was previously 32 bit, and was upgraded to 64 bit, with the agent pushed from backup exec onto the machine after it joined our domain again under the same computer name. AOFO was installed on the machine and checked in the options.

Screenshot:

Job Log for Cucumber Full


--------------------------------------------------------------------------------


Completed status: Failed    See error(s)  
Expand AllCollapse All  


Job Information
Job server: BACKUP_SERVER
Job name: Cucumber Full
Job started: Sunday, August 19, 2012 at 2:00:04 PM
Job type: Backup
Job Log: BEX_BACKUP_SERVER_21590.xml

 

Drive and media mount requested: 8/19/2012 2:00:04 PM

Device and Media Information
Drive and media information from media mount: 8/19/2012 2:00:48 PM
Robotic Library Name: OVERLAND 1
Drive Name: HP 4
Slot: 22
Media Label: 000029L4
Media GUID: {3510286e-67ed-4594-a458-331c80e47e4c}
Overwrite Protected Until: 8/19/2013 3:50:23 AM
Appendable Until: 12/30/9999 5:00:00 PM
Targeted Media Set Name: SRFS_Internal

All Media Used
 000029L4

 


Job Operation - Backup  
Backup Options
Media operation - Append to media, overwrite if no appendable media is available.
Compression Type: Hardware [if available, otherwise none]
Encryption Type: None

 



Server - REMOTE_COMP
AOFO: Started for resource: "\\REMOTE_COMP\C:". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
The snapshot provider used by VSS for volume C: - Microsoft Software Shadow Copy provider 1.0 (Version 1.0.0.7).
Network control connection is established between [FE80::6598:A9AE:1D13:4DA5%11]:63710 <--> [FE80::CC04:75A9:DFCD:497A%11]:10000
Network data connection is established between    [FE80::6598:A9AE:1D13:4DA5%11]:63717 <--> [FE80::CC04:75A9:DFCD:497A]:52616

Set Information - \\REMOTE_COMP\C:
Backup Set Information
Family Name: "Media created 7/17/2012 9:14:35 AM"
Backup of "\\REMOTE_COMP\C:"
Backup set #492 on storage media #12
Backup set description: "Cucumber Full"
Backup Method: Full - Back Up Files - Reset Archive Bit

 


Backup started on 8/19/2012 at 2:03:19 PM.

Backup Set Detail Information
Unable to open the item \\REMOTE_COMP\C:\ProgramData\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\CmnClnt\_lck\_RDRPluginG - skipped.
Unable to open the item \\REMOTE_COMP\C:\ProgramData\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\CmnClnt\_lck\_SNDPluginG - skipped.

 

Backup completed on 8/19/2012 at 2:46:08 PM.

Backup Set Summary
Backed up 83862 files in 11441 directories.
2 items were skipped.
Processed 25,389,598,250 bytes in  42 minutes and  49 seconds.
Throughput rate: 566 MB/min
Compression Type: Hardware

 

 


AOFO: Started for resource: "\\REMOTE_COMP\D:". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).
The snapshot provider used by VSS for volume D: - Microsoft Software Shadow Copy provider 1.0 (Version 1.0.0.7).
Network control connection is established between [FE80::6598:A9AE:1D13:4DA5%11]:63710 <--> [FE80::CC04:75A9:DFCD:497A%11]:10000
Network data connection is established between    [FE80::6598:A9AE:1D13:4DA5%11]:63825 <--> [FE80::CC04:75A9:DFCD:497A]:49957

Set Information - \\REMOTE_COMP\D:
Backup Set Information
Family Name: "Media created 7/17/2012 9:14:35 AM"
Backup of "\\REMOTE_COMP\D:"
Backup set #493 on storage media #12
Backup set description: "Cucumber Full"
Backup Method: Full - Back Up Files - Reset Archive Bit

 


Backup started on 8/19/2012 at 2:46:13 PM.
Backup completed on 8/19/2012 at 2:58:41 PM.

Backup Set Summary
Backed up 16820 files in 1551 directories.
Processed 8,247,183,525 bytes in  12 minutes and  28 seconds.
Throughput rate: 631 MB/min
Compression Type: Hardware

 

 

Note: Resource: "\\REMOTE_COMP\Utility Partition" is not snappable. Attempting to back up directly from the source volume.
Network control connection is established between [FE80::6598:A9AE:1D13:4DA5%11]:63710 <--> [FE80::CC04:75A9:DFCD:497A%11]:10000
Network data connection is established between    [FE80::6598:A9AE:1D13:4DA5%11]:63857 <--> [FE80::CC04:75A9:DFCD:497A]:49961

Set Information - \\REMOTE_COMP\Utility Partition
Backup Set Information
Family Name: "Media created 7/17/2012 9:14:35 AM"
Backup of "\\REMOTE_COMP\Utility Partition"
Backup set #494 on storage media #12
Backup set description: "Cucumber Full"
Backup Method: Full - Back Up Files - Reset Archive Bit

 


Backup started on 8/19/2012 at 2:58:42 PM.
Backup completed on 8/19/2012 at 2:59:10 PM.

Backup Set Summary
Backed up 1 Utility Partitions
Processed 172740789 bytes in  28 seconds.
Throughput rate: 353 MB/min
Compression Type: Hardware

 

 

Note: Resource: "\\REMOTE_COMP\System?State" is not snappable. Attempting to back up directly from the source volume.
Network control connection is established between [FE80::6598:A9AE:1D13:4DA5%11]:63710 <--> [FE80::CC04:75A9:DFCD:497A%11]:10000
Network data connection is established between    [FE80::6598:A9AE:1D13:4DA5%11]:63860 <--> [FE80::CC04:75A9:DFCD:497A]:49962
V-79-57344-65218 - A failure occurred accessing the SnapShot handle.


 

 


Job Operation - Verify  

Server - REMOTE_COMP

Set Information - \\REMOTE_COMP\C:  
Verify Set Information
Verify of "\\REMOTE_COMP\C: "
Backup set #492 on storage media #12
Backup set description: "Cucumber Full"

 


Verify started on 8/19/2012 at 3:03:21 PM.
Verify completed on 8/19/2012 at 3:06:55 PM.

Verify Set Summary
Verified 83862 files in 11441 directories.
Processed 25,389,598,250 bytes in  3 minutes and  34 seconds.
Throughput rate: 6789 MB/min

 

 


Set Information - \\REMOTE_COMP\D:  
Verify Set Information
Verify of "\\REMOTE_COMP\D: "
Backup set #493 on storage media #12
Backup set description: "Cucumber Full"

 


Verify started on 8/19/2012 at 3:06:55 PM.
Verify completed on 8/19/2012 at 3:08:03 PM.

Verify Set Summary
Verified 16820 files in 1551 directories.
Processed 8,247,183,525 bytes in  1 minute and  8 seconds.
Throughput rate: 6940 MB/min

 

 


Set Information - \\REMOTE_COMP\Utility Partition  
Verify Set Information
Verify of "\\REMOTE_COMP\Utility Partition "
Backup set #494 on storage media #12
Backup set description: "Cucumber Full"

 


Verify started on 8/19/2012 at 3:08:03 PM.
Verify completed on 8/19/2012 at 3:08:05 PM.

Verify Set Summary
Verified 1 Utility Partitions
Processed 172740789 bytes in  2 seconds.
Throughput rate: 4942 MB/min

 

 

 

 


Job Completion Status
Job ended: Sunday, August 19, 2012 at 3:10:28 PM
Completed status: Failed
Final error: 0xe000fec2 - A failure occurred accessing the SnapShot handle.
Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-65218

 

Errors
Click an error below to locate it in the job log
BackupV-79-57344-65218 - A failure occurred accessing the SnapShot handle.

 

Exceptions
Click an exception below to locate it in the job log
Backup- \\REMOTE_COMP\C:Unable to open the item \\REMOTE_COMP\C:\ProgramData\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\CmnClnt\_lck\_RDRPluginG - skipped.
Unable to open the item \\REMOTE_COMP\C:\ProgramData\Symantec\Symantec Endpoint Protection\CurrentVersion\Data\CmnClnt\_lck\_SNDPluginG - skipped.

 
 

[marcusolini]

Hi.  Thank you for the detailed information.  It appears that System State is reporting that a snapshot is not required, which is unexpected.  Debug logs from the Backup Exec Media Server Engine and Remote Server Remote Agent will be needed to diagnose the issue.  When able, please collect debug logs using procedures that you are already familiar with or using the below procedures.   When complete, please attach the files to this post for review.  Thank you.

Please collect debug output by using the Backup Exec Debug Monitor for the failing backup job…

(1) Enable the Backup Exec Debug Monitor, by either...

(a) Executing the SGMON.EXE process from within the Backup Exec installation directory.  Example: C:\Program Files\Symantec\Backup Exec\Sgmon.exe.  If a remote system that is not the BE Media Server, then can copy SGMON.EXE and SGMONCLIENT.DLL to the system; if not present   AND\OR

(b) Clicking ‘Tools’ from the menu bar (Alt-t) and selecting ‘Support Utilities’ and clicking ‘Run the Debug Monitor for active debugging’; if a BE Media Server.

(2) Select ‘Job Engine, RAWS, Agent Browser’ from the ‘Capture’ section (Alt-j).

(3) Select ‘Capture to file’ from the ‘Capture to file:’ section (Alt-o).

(4) Run the backup job that is failing with error:  ‘V-79-57344-65218 - A failure occurred accessing the SnapShot handle’.

(5) Save the debug log outputs  (Alt-n).

[Mgamerz]

I attached the logs from both computers as you described. The backup is from a different failing computer with the same issue. I have replaced the computer names with generic text. The server is BESERVER and the computer that is failing is REMOTE_COMP.

I saw some messages about a failure with VSS on the System State.

Thanks,

Michael

[marcusolini · Accepted solution]

Hi.  Thank you for collecting the requested information so quickly!  The debug logs indicate that the System State selection in the job is for the historical Windows 2000 (and Windows XP 32-bit) System State, which does not use snapshot technology for backup.  The System State for Windows 2003 (and Windows XP 64-bit) and newer Windows operating systems requires snapshot technology.  With this said, I would speculate that the systems that have been upgraded were upgraded from Windows XP 32-bit to Windows 7 64-bit.   If this theory is correct, then new backup jobs will need to be created for the affected systems (and the historic backup jobs deleted).  From your post it appears that this approach has been attempted so this is puzzling.  When able, please research the upgrade that was performed and\or create a new backup job for the affected system that corresponds to the debug logs that were provided.  Please let me know the outcome so we can determine if further investigation is required.  Thank you.

[Mgamerz]

I see. Some of the computers were XP at one time (the jobs I don't think had changed), and then it switched to Windows 7 32 bit (We weren't allowed to use 64 bit until management OK'd it), and now we are changing everything over to 64 bit. I'll recreate the jobs again and see if that works. Would it be possible for it to change the snapshot technology to adapt to items like this in the future?

Thanks,

Michael

[Mgamerz]

Deleted all jobs for one that I tested this morning that fialed (the one I sent you). Recreated the jobs and ran an incremental. The snapshot is still failing on the backup of the System State.

Error                  : e000fec2 - A failure occurred accessing the SnapShot handle.

Agent used                     : Yes

Advanced Open File Option used : No
 

I did not have AOFO option checked in the preferences. (changed from original post)

It appears backup exec is trying to use VSS anyways.

The computer was previously a Dell Latitude D630 (with XP) and it has been replaced with a Dell Latitude E6420 (Windows 7 64 bit). The name of the computer and active directory accounts are the same still.

I was the one who installed Windows 7 onto it (we have an enterprise version we use) and it seems to be happening to all machines I have been upgrading, however the new machines that I add that did not previously have a job with them seem to work fine.

I do not know if a selection list has more than just text of the selections, but I am now going to use the resources browser to get the items to back up from the computer. The last test I did in this post was with a selection list that was made when the computer was still on XP.

UPDATE: When I use the resources browser I now see Shadow Copy Components as a resource.

UPDATE 2: I am a bit confused, because from some posts I just researched through Google, I found that Shadow Copy Components includes the System State. I also found a thread on technet that seems very relevant, but doesn't provide a solution.

http://social.technet.microsoft.com/Forums/en-US/smallbusinessserver/thread/887451be-0e3f-4697-b807-...

I'm trying out a few configurations right now to see if any of them complete without the failure message.

POSSIBLE RESOLUTION UPDATE: When I ticked the Shadow Copy Components, it completed with exceptions, missing the oem35.inf that never makes it into any backups it seems.

VSS Snapshot warning. File c:\windows\inf\oem35.inf is not present on the snapshot.
 

[marcusolini]

Hi.  When able, please confirm if the suggestion to create a new backup job (and selection list) has corrected the originally reported backup failure for System State: ‘V-79-57344-65218 - A failure occurred accessing the SnapShot handle’.  If so, please apply this suggestion to the other backups that are experiencing this same specific backup failure to ensure that this is a valid solution.  We can discuss and investigate the new issue that is being experienced once we have addressed the originally reported backup failure.  Thank you.

[Eddie_Ham]

Hi marcusolini, I am having a very similar proble with the e000fec2 snapshot handle error. I simply removed B2T and went to a B2D solution on an msa1000. i am running be12.5 on a win2003 srvr. My monthly and weekly fulls complete without error however the daily incremental fails consistently. Can you help??? PLeasee!!!

[marcusolini]

Hi Eddie.  Sure.  When able, please similarly capture the debug logs from the Backup Exec Media Server Engine and Remote Server Remote Agent.  When complete, please attach the job log and the debug log files to this post for review. Thank you.

[Mgamerz]

Just so people who find this thread know, I did find a solution.

What I did was go to the selection list for the failing job (snapshot handle error), and delete all items in it, and then browsed to that computer in the resource explorer and reticked it. It seems XP and 7 have different VSS setups (XP had 3, 7 had 4).

[Eddie_Ham]

Hi Marcusolini,

I am posting the job log and debug log. i hope you can help. this is driving me crazy.

This is the DEBUG LOG

***************************************************************************
*****                  SGMon log for \\BKUP01                         *****
***************************************************************************
BENGINE:  [11/20/12 09:40:22] [5728]     PDI processor not needed, performing traditional backup
BENGINE:  [11/20/12 09:40:22] [5728]     NDMPRAHostBackupEngine::CreateBSDProcessor() Instantiating an offhost backup bsd processor. It is not a GRT Resource:
BENGINE:  [11/20/12 09:40:22] [5728]     Failed to Load AllowExcludes key, We will update DR file evenif there are userExcludes. Error: 5
BENGINE:  [11/20/12 09:40:22] [5728]     Exit NDMPSnapHostBackupEngine::CreateBSDProcessor
BENGINE:  [11/20/12 09:40:22] [5728]     Enter NDMPSnapHostBackupEngine::SnapSingleVolume
BENGINE:  [11/20/12 09:40:22] [5728]     Exit NDMPSnapHostBackupEngine::SnapSingleVolume
BENGINE:  [11/20/12 09:40:22] [5728]     DeviceIsNT4Server: agent info: [Major Version=5 Minor Version=2 Build Number=3790 ServicePack Major=2 ServicePack Minor=0 SuiteMask=274 ProductType=3 ProcessorType=AMD64].
BENGINE:  [11/20/12 09:40:22] [5728]     DeviceIsIA64Server: agent info: [Major Version=5 Minor Version=2 Build Number=3790 ServicePack Major=2 ServicePack Minor=0 SuiteMask=274 ProductType=3 ProcessorType=AMD64].
BENGINE:  [11/20/12 09:40:22] [5728]     Failed to open Microsoft cluster (sql01.ad.iss-eps.net)
BENGINE:  [11/20/12 09:40:22] [5728]     Enter OffhostBackupBSDProcessor::UpdateConnectionSetup
BENGINE:  [11/20/12 09:40:22] [5728]     Exit OffhostBackupBSDProcessor::UpdateConnectionSetup
BENGINE:  [11/20/12 09:40:22] [5728]     TF_OpenSet( )
BENGINE:  [11/20/12 09:40:22] [5728]     Requested Set: ID = ffffffff Seq = -1 Set = -1
BENGINE:  [11/20/12 09:40:22] [5728]     Current VCB: ID = 41744c59  Seq = 1  Set = 5
BENGINE:  [11/20/12 09:40:22] [5728]     PositionAtSet( :( TF Msg = d
BENGINE:  [11/20/12 09:40:22] [5728]       UI Msg = 800f
BENGINE:  [11/20/12 09:40:22] [5728]     HARDWARE COMPRESSION ===> Compression is NOT configurable.
BENGINE:  [11/20/12 09:40:22] [5728]     HARDWARE ENCRYPTION ===> Encryption is NOT supported.
BENGINE:  [11/20/12 09:40:22] [5728]     GET_DRV_INF: bsize = 1024
BENGINE:  [11/20/12 09:40:22] [5728]     GET_DRV_INF: preferredBlockSize = 1024
BENGINE:  [11/20/12 09:40:22] [5728]     Deleting left over Set Map!!!
BENGINE:  [11/20/12 09:40:22] [5728]     F40_SeekEOD starting seek...
BENGINE:  [11/20/12 09:40:22] [5728]     F40_SeekEOD calling OTC_GetPrevSM with expect_sm = 1 ...
BENGINE:  [11/20/12 09:40:22] [5728]     OTC_GetPrevSM spacing back two filemarks...
BENGINE:  [11/20/12 09:40:22] [5728]     OTC_GetPrevSM: ESET position = 4686209
BENGINE:  [11/20/12 09:40:22] [5728]     OTC_GetPrevSM calling OTC_SMtoFile...
BENGINE:  [11/20/12 09:40:22] [5728]     SetupFormatEnv( fmt=0 )
BENGINE:  [11/20/12 09:40:22] [5728]     End of TF_OpenSet: Ret_val = 0x00000000 Buffs = 2 HiWater = 7
BENGINE:  [11/20/12 09:40:22] [5728]     PDIBackupBSDProcessor::InitSet( ). Tape or CPS Incr. Using base class
BENGINE:  [11/20/12 09:40:22] [5728]     B2D Buffered Writes ===> Setting buffering to default.
BENGINE:  [11/20/12 09:40:22] [5728]     Current Block is = 47818f
BENGINE:  [11/20/12 09:40:22] [5728]     Media Server to initiate connection for data transfer
BENGINE:  [11/20/12 09:40:22] [5728]     TF_InitMediaServerReverseConnection : Data Connection information: A connection was established between end-points 10.1.1.45:1167 and 10.1.1.25:1580.
BENGINE:  [11/20/12 09:40:23] [5728]     

    dataStartBackup: ndmpSendRequest returned: 0x0, 0
BENGINE:  [11/20/12 09:40:23] [5728]      data halted: SUCCESSFUL
BENGINE:  [11/20/12 09:40:23] [5728]     BaseBSDProcessor::PollDataServer: Shutting down.
BENGINE:  [11/20/12 09:40:23] [5728]     NDMPBackupEngine::ProcessBSD: We received a successful notify_data_halted without receiving a MSG_START_BACKUP_SET.
BENGINE:  [11/20/12 09:40:23] [5728]     ERROR: 13 Error: Invalid state to process request
BENGINE:  [11/20/12 09:40:23] [5728]     *** getLastNDMPError Calling to get last NDMP Error.
BENGINE:  [11/20/12 09:40:23] [5728]     Enter OffhostBackupBSDProcessor::HandleOffhostCompletionStatus
BENGINE:  [11/20/12 09:40:23] [5728]     Exit OffhostBackupBSDProcessor::HandleOffhostCompletionStatus
BENGINE:  [11/20/12 09:40:23] [5728]     BackupMessageHandler::CloseCatalog called
BENGINE:  [11/20/12 09:40:23] [5728]     Not: 'm_bCatalogDrivenBackup && CatBuildVector[engine_id] && GetCarryFwdCatalogInfoForRemainingObjs( )'
BENGINE:  [11/20/12 09:40:23] [5728]     ERROR: CatBuildVector[engine_id] is NULL!!!!!
BENGINE:  [11/20/12 09:40:23] [5728]         Info: BackupBSDProcessor::WriteEndSet() ... TFLE_UNEXPECTED_EOS
BENGINE:  [11/20/12 09:40:23] [5728]     TF_FreeTapeBuffers: from 0 to 0 buffers
BENGINE:  [11/20/12 09:40:23] [5728]     TF_CloseSet
BENGINE:  [11/20/12 09:40:23] [5728]     Enter OffhostBackupBSDProcessor::ResetConnectionSetup
BENGINE:  [11/20/12 09:40:23] [5728]     Exit OffhostBackupBSDProcessor::ResetConnectionSetup
BENGINE:  [11/20/12 09:40:23] [5728]     Enter NDMPSnapHostBackupEngine::TrackSnappedBSD
BENGINE:  [11/20/12 09:40:23] [5728]     Exit NDMPSnapHostBackupEngine::TrackSnappedBSD
BENGINE:  [11/20/12 09:40:23] [5728]     Exit NDMPSnapHostBackupEngine::ProcessBSDs
BENGINE:  [11/20/12 09:40:23] [5728]     NDMPAgentConnector::EndConnection(): Data Server = BKUP01
BEREMOTE: [11/20/12 09:40:23] [5352]     @@@@@@@MyCloseSocket called with sockfd = 2196(0x894)    retval = 0
BENGINE:  [11/20/12 09:40:23] [5728]     NDMPAgentConnector::EndConnection(): Data Server = sql01.ad.iss-eps.net
BENGINE:  [11/20/12 09:40:23] [5728]     TF_PutChannel( ) 03C579D0
BENGINE:  [11/20/12 09:40:23] [5728]     TAPEALERT: Get TapeAlert Flags Return Code = 0X0
BENGINE:  [11/20/12 09:40:23] [5728]     TAPEALERT: TapeAlert Device Flag  = 0X0
BENGINE:  [11/20/12 09:40:23] [5728]     TAPEALERT: TapeAlert Changer Flag = 0X0
BENGINE:  [11/20/12 09:40:23] [5728]     TAPEALERT: Get TapeAlert Flags Return Code = 0X0
BENGINE:  [11/20/12 09:40:23] [5728]     TAPEALERT: Get TapeAlert Flags Return Code = 0X0
BENGINE:  [11/20/12 09:40:23] [5728]     Updating session {3631b25b-4113-4a29-9754-531fbd2aa8fc} with drive HPEPS 2 {19b1819d-c42b-4b7c-a315-35c7d76cd9d1}.1
BENGINE:  [11/20/12 09:40:23] [5728]     TF_OpenTape()
BENGINE:  [11/20/12 09:40:23] [5728]     TF_GetChannel( ) reusing channel 015E08B0
BENGINE:  [11/20/12 09:40:23] [5728]     Allocated 10 buffers, size 32768 bytes, total used: 328880
BENGINE:  [11/20/12 09:40:23] [5728]     HARDWARE COMPRESSION ===> Compression is NOT configurable.
BENGINE:  [11/20/12 09:40:23] [5728]     HARDWARE ENCRYPTION ===> Encryption is NOT supported.
BENGINE:  [11/20/12 09:40:23] [5728]     GET_DRV_INF: bsize = 1024
BENGINE:  [11/20/12 09:40:23] [5728]     GET_DRV_INF: preferredBlockSize = 1024
BENGINE:  [11/20/12 09:40:23] [5728]     TF_OpenSet( )
BENGINE:  [11/20/12 09:40:23] [5728]     Requested Set: ID = 41744c59 Seq = 1 Set = 1
BENGINE:  [11/20/12 09:40:23] [5728]     Current VCB: ID = 41744c59  Seq = 1  Set = 5
BENGINE:  [11/20/12 09:40:23] [5728]     End of TF_OpenSet: Ret_val = 0x00000000 Buffs = 10 HiWater = 7
BENGINE:  [11/20/12 09:40:23] [5728]     GotoBlock: VCB_PBA = 3, VCB_LBA = 0, LBA = 0,
BENGINE:  [11/20/12 09:40:23] [5728]                PBA = 3, offset = 0
BENGINE:  [11/20/12 09:40:23] [5728]     Set 1 was written by version 12.5.
BENGINE:  [11/20/12 09:40:25] [4904]     Updating status for: 'sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup' (0x4 0x0)
BENGINE:  [11/20/12 09:40:25] [4904]     Status for: 'sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup' updated
BENGINE:  [11/20/12 09:40:41] [5728]     Device Error = 1101
BENGINE:  [11/20/12 09:40:41] [5728]     Len Req = 32768 Len Got = 2048
BENGINE:  [11/20/12 09:40:41] [5728]     TF xfer time = 17 seconds.
BENGINE:  [11/20/12 09:40:41] [5728]     TF_CloseSet
BENGINE:  [11/20/12 09:40:41] [5728]     TF_OpenSet( )
BENGINE:  [11/20/12 09:40:41] [5728]     Requested Set: ID = 41744c59 Seq = 1 Set = 2
BENGINE:  [11/20/12 09:40:41] [5728]     Current VCB: ID = 41744c59  Seq = 1  Set = 1
BENGINE:  [11/20/12 09:40:41] [5728]     End of TF_OpenSet: Ret_val = 0x00000000 Buffs = 10 HiWater = 7
BENGINE:  [11/20/12 09:40:41] [5728]     GotoBlock: VCB_PBA = 487472, VCB_LBA = 0, LBA = 0,
BENGINE:  [11/20/12 09:40:41] [5728]                PBA = 487472, offset = 0
BENGINE:  [11/20/12 09:40:41] [5728]     Device Error = 1101
BENGINE:  [11/20/12 09:40:41] [5728]     Len Req = 32768 Len Got = 13312
BENGINE:  [11/20/12 09:40:41] [5728]     Set 2 was written by version 12.5.
BENGINE:  [11/20/12 09:40:41] [5728]     TF xfer time = 0 seconds.
BENGINE:  [11/20/12 09:40:41] [5728]     TF_CloseSet
BENGINE:  [11/20/12 09:40:41] [5728]     TF_OpenSet( )
BENGINE:  [11/20/12 09:40:41] [5728]     Requested Set: ID = 41744c59 Seq = 1 Set = 3
BENGINE:  [11/20/12 09:40:41] [5728]     Current VCB: ID = 41744c59  Seq = 1  Set = 2
BENGINE:  [11/20/12 09:40:41] [5728]     End of TF_OpenSet: Ret_val = 0x00000000 Buffs = 10 HiWater = 7
BENGINE:  [11/20/12 09:40:41] [5728]     GotoBlock: VCB_PBA = 487498, VCB_LBA = 0, LBA = 0,
BENGINE:  [11/20/12 09:40:41] [5728]                PBA = 487498, offset = 0
BENGINE:  [11/20/12 09:40:41] [5728]     Set 3 was written by version 12.5.
BENGINE:  [11/20/12 09:40:44] [5728]     Device Error = 1101
BENGINE:  [11/20/12 09:40:44] [5728]     Len Req = 32768 Len Got = 19456
BENGINE:  [11/20/12 09:40:44] [5728]     TF xfer time = 2 seconds.
BENGINE:  [11/20/12 09:40:44] [5728]     TF_CloseSet
BENGINE:  [11/20/12 09:40:44] [5728]     TF_OpenSet( )
BENGINE:  [11/20/12 09:40:44] [5728]     Requested Set: ID = 41744c59 Seq = 1 Set = 4
BENGINE:  [11/20/12 09:40:44] [5728]     Current VCB: ID = 41744c59  Seq = 1  Set = 3
BENGINE:  [11/20/12 09:40:44] [5728]     End of TF_OpenSet: Ret_val = 0x00000000 Buffs = 10 HiWater = 7
BENGINE:  [11/20/12 09:40:44] [5728]     GotoBlock: VCB_PBA = 564716, VCB_LBA = 0, LBA = 0,
BENGINE:  [11/20/12 09:40:44] [5728]                PBA = 564716, offset = 0
BENGINE:  [11/20/12 09:40:44] [5728]     Device Error = 1101
BENGINE:  [11/20/12 09:40:44] [5728]     Len Req = 32768 Len Got = 22528
BENGINE:  [11/20/12 09:40:44] [5728]     Set 4 was written by version 12.5.
BENGINE:  [11/20/12 09:40:44] [5728]     TF xfer time = 0 seconds.
BENGINE:  [11/20/12 09:40:44] [5728]     TF_CloseSet
BENGINE:  [11/20/12 09:40:44] [5728]     TF_OpenSet( )
BENGINE:  [11/20/12 09:40:44] [5728]     Requested Set: ID = 41744c59 Seq = 1 Set = 5
BENGINE:  [11/20/12 09:40:44] [5728]     Current VCB: ID = 41744c59  Seq = 1  Set = 4
BENGINE:  [11/20/12 09:40:44] [5728]     End of TF_OpenSet: Ret_val = 0x00000000 Buffs = 10 HiWater = 7
BENGINE:  [11/20/12 09:40:44] [5728]     GotoBlock: VCB_PBA = 564753, VCB_LBA = 0, LBA = 0,
BENGINE:  [11/20/12 09:40:44] [5728]                PBA = 564753, offset = 0
BENGINE:  [11/20/12 09:40:44] [5728]     Set 5 was written by version 12.5.
BENGINE:  [11/20/12 09:43:07] [5728]     Device Error = 1101
BENGINE:  [11/20/12 09:43:07] [5728]     Len Req = 32768 Len Got = 15360
BENGINE:  [11/20/12 09:43:07] [5728]     TF xfer time = 143 seconds.
BENGINE:  [11/20/12 09:43:07] [5728]     TF_CloseSet
BENGINE:  [11/20/12 09:43:07] [5728]     TF_PutChannel( ) 015E08B0
BENGINE:  [11/20/12 09:43:07] [3876]     DeviceManager: incoming event fired
BENGINE:  [11/20/12 09:43:07] [3876]     TF_FreeDriveContext( 3CCCF40 )
BENGINE:  [11/20/12 09:43:07] [3876]     TF_FreeTapeBuffers: from 10 to 0 buffers
BENGINE:  [11/20/12 09:43:07] [3876]     FreeFormatEnv( cur_fmt=0 )
BENGINE:  [11/20/12 09:43:07] [3876]     DeviceManager: processing pending requests
BENGINE:  [11/20/12 09:43:07] [5728]     Removing 'sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup' from status update list
BENGINE:  [11/20/12 09:43:07] [3876]     DeviceManager: going to sleep for 900000 msecs
BENGINE:  [11/20/12 09:43:07] [5728]     Updating status for: 'sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup' (0x18 0x0)
BENGINE:  [11/20/12 09:43:07] [3876]     DeviceManager: dismount event fired
BENGINE:  [11/20/12 09:43:07] [3876]     DeviceManager: processing pending requests
BENGINE:  [11/20/12 09:43:07] [3876]     DeviceManager: going to sleep for 900000 msecs
BENGINE:  [11/20/12 09:43:07] [5728]     Status for: 'sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup' updated
BENGINE:  [11/20/12 09:43:07] [5728]     BackupJob::MergeBEVSRJobLogsIfNecessary: No VSR log file found, no merging necessary
BENGINE:  [11/20/12 09:43:07] [5728]     Ending job 'sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup' with error status (-536805694)
BENGINE:  [11/20/12 09:43:07] [5728]     Job thread terminating
BESERVER: [11/20/12 09:43:08] [5472]     "Cluster" key does not appear to be present in the registry
BESERVER: [11/20/12 09:43:08] [5472]     Failed to open Microsoft cluster ()
BESERVER: [11/20/12 09:43:08] [5472]     VCS cluster keys do not appear to be present in the registry
BESERVER: [11/20/12 09:43:08] [5472]     Failed to open VCS cluster ()
BKUPEXEC: [11/20/12 09:45:53] [0000]     UIVirtualObjectView(JobHistoryView) Queried 38 items: 0 to 37
BENETNS:  [11/20/12 09:48:54] [4420]     Beginning agent database grooming.
BENETNS:  [11/20/12 09:48:54] [4420]     Agent database grooming complete.

This is the JOB LOG

Job Log for sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup


--------------------------------------------------------------------------------


Completed status: Failed  
Expand AllCollapse All  


Job Information
Job server: BKUP01
Job name: sql01-sql rotation-sql Daily Incremental Backup1-sql rotation-sql Daily Incremental Backup
Job started: Tuesday, November 20, 2012 at 9:34:45 AM
Job type: Backup
Job Log: BEX_BKUP01_06386.xml

 

Drive and media mount requested: 11/20/2012 9:34:45 AM

Device and Media Information
Drive and media information from media mount: 11/20/2012 9:34:48 AM
Drive Name: HPEPS 2
Media Label: B2D000049
Media GUID: {2cab7b31-8bfe-4d88-882c-7895350823b9}
Overwrite Protected Until: 12/18/2012 9:34:48 AM
Appendable Until: 12/30/9999 6:00:00 PM
Targeted Media Set Name: sql01

All Media Used
 B2D000049


 


Job Operation - Backup  
Backup Options
Media operation - overwrite.

Compression Type: Hardware [if available, otherwise none]

Encryption Type: None


 



Server - sql01.ad.iss-eps.net
ADBO: The Advanced Disk-based Backup Option was selected for this job. Data is being backed up using offhost backup.
ADBO: The Advanced Disk-based Backup Option was selected for this job. Data is being backed up using offhost backup.
V-79-57344-34100 - ADBO: Offhost backup initialization failure on: "\\sql01.ad.iss-eps.net\C:".
Snapshot provider error (0xE0008534): Backup Exec could not find a Microsoft Volume Shadow Copy Services (VSS) software or hardware provider on the media server. To use offhost backup, a VSS provider must be installed on both the media server and on the remote computer.

Check the Windows Event Viewer for details.

ADBO: Offhost backup failed. Attempting to back up volumes directly from the remote computer.
Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1109 <--> 10.1.1.25:1494

Set Information - \\sql01.ad.iss-eps.net\C:  
Backup Set Information
Family Name: "Media created 11/20/2012 9:34:45 AM"
Backup of "\\sql01.ad.iss-eps.net\C:" as ""
Backup set #1 on storage media #1
Backup set description: "sql Daily Incremental Backup"
Backup Method: Incremental - Changed Files - Reset Archive Bit

 


Backup started on 11/20/2012 at 9:34:54 AM.
Backup completed on 11/20/2012 at 9:35:26 AM.

Backup Set Summary
Backed up 97 files in 10708 directories.
Processed 489776197 bytes in  32 seconds.
Throughput rate: 876 MB/min
Compression Type: None

 

 

Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1117 <--> 10.1.1.25:1511

Set Information - \\sql01.ad.iss-eps.net\D:  
Backup Set Information
Family Name: "Media created 11/20/2012 9:34:45 AM"
Backup of "\\sql01.ad.iss-eps.net\D:" as ""
Backup set #2 on storage media #1
Backup set description: "sql Daily Incremental Backup"
Backup Method: Incremental - Changed Files - Reset Archive Bit

 


Backup started on 11/20/2012 at 9:35:29 AM.
Backup completed on 11/20/2012 at 9:35:29 AM.

Backup Set Summary
Backed up 0 files in 7 directories.
Processed 5,460 bytes in  1 second.
Throughput rate: 0.312 MB/min
Compression Type: None

 


This backup set may not contain any data.

 

Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1119 <--> 10.1.1.25:1519

Set Information - \\sql01.ad.iss-eps.net\E:  
Backup Set Information
Family Name: "Media created 11/20/2012 9:34:45 AM"
Backup of "\\sql01.ad.iss-eps.net\E:" as ""
Backup set #3 on storage media #1
Backup set description: "sql Daily Incremental Backup"
Backup Method: Incremental - Changed Files - Reset Archive Bit

 


Backup started on 11/20/2012 at 9:35:31 AM.
Backup completed on 11/20/2012 at 9:35:36 AM.

Backup Set Summary
Backed up 5 files in 17 directories.
Processed 79,036,032 bytes in  5 seconds.
Throughput rate: 904 MB/min
Compression Type: None

 

 

Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1120 <--> 10.1.1.25:1525

Set Information - \\sql01.ad.iss-eps.net\F:  
Backup Set Information
Family Name: "Media created 11/20/2012 9:34:45 AM"
Backup of "\\sql01.ad.iss-eps.net\F:" as ""
Backup set #4 on storage media #1
Backup set description: "sql Daily Incremental Backup"
Backup Method: Incremental - Changed Files - Reset Archive Bit

 


Backup started on 11/20/2012 at 9:35:38 AM.
Backup completed on 11/20/2012 at 9:35:38 AM.

Backup Set Summary
Backed up 0 files in 16 directories.
Processed 7,788 bytes in  1 second.
Throughput rate: 0.446 MB/min
Compression Type: None

 


This backup set may not contain any data.

 

Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1123 <--> 10.1.1.25:1530

Database Consistency Checks
Consistency checking eps-net
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking eps-vcenter
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking eps-vum
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking eXpress
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking gwlm
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking hpio
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking hpvmmsqldb
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking Insight_v50_0_1730363
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking lsdt
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking master
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking model
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking msdb
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking PMP_V5_0
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking VI_VC
Consistency Check Physical Only.
Consistency check was completed.
Consistency checking VirtualManagerDB
Consistency Check Physical Only.
Consistency check was completed.

 


Set Information - sql01.ad.iss-eps.net  
Backup Set Information
Family Name: "Media created 11/20/2012 9:34:45 AM"
Backup of "sql01.ad.iss-eps.net" as ""
Backup set #5 on storage media #1
Backup set description: "sql Daily Incremental Backup"
Backup Method: Full - Back up entire database or filegroup
Microsoft SQL Server Agent: Started

 


Backup started on 11/20/2012 at 9:37:16 AM.

Backup Set Detail Information

 

Backup completed on 11/20/2012 at 9:40:19 AM.

Backup Set Summary
Backed up 15 databases
Processed 4,216,934,830 bytes in  3 minutes and  3 seconds.
Throughput rate: 1319 MB/min
Compression Type: None



Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1165 <--> 10.1.1.25:1573
Network control connection is established between 10.1.1.45:1099 <--> 10.1.1.25:10000
Network data connection is established between    10.1.1.45:1167 <--> 10.1.1.25:1580

 
Job Operation - Verify  

Set Information - \\sql01.ad.iss-eps.net\C: system
Verify Set Information
Verify of "\\sql01.ad.iss-eps.net\C: system"
Backup set #1 on storage media #1
Backup set description: "sql Daily Incremental Backup"

 


Verify started on 11/20/2012 at 9:40:23 AM.
Verify completed on 11/20/2012 at 9:40:41 AM.

Verify Set Summary
Verified 97 files in 10708 directories.
Processed 489776197 bytes in  18 seconds.
Throughput rate: 1557 MB/min

 
Set Information - \\sql01.ad.iss-eps.net\D: sql_logs
Verify Set Information
Verify of "\\sql01.ad.iss-eps.net\D: sql_logs"
Backup set #2 on storage media #1
Backup set description: "sql Daily Incremental Backup"


Verify started on 11/20/2012 at 9:40:41 AM.
Verify completed on 11/20/2012 at 9:40:41 AM.

Verify Set Summary
Verified 0 files in 7 directories.
Processed 5,460 bytes in  1 second.
Throughput rate: 0.312 MB/min


Set Information - \\sql01.ad.iss-eps.net\E: sql_data_0
Verify Set Information
Verify of "\\sql01.ad.iss-eps.net\E: sql_data_0"
Backup set #3 on storage media #1
Backup set description: "sql Daily Incremental Backup"


Verify started on 11/20/2012 at 9:40:41 AM.
Verify completed on 11/20/2012 at 9:40:44 AM.

Verify Set Summary
Verified 5 files in 17 directories.
Processed 79,036,032 bytes in  3 seconds.
Throughput rate: 1507 MB/min


Set Information - \\sql01.ad.iss-eps.net\F: sql_data_1
Verify Set Information
Verify of "\\sql01.ad.iss-eps.net\F: sql_data_1"
Backup set #4 on storage media #1
Backup set description: "sql Daily Incremental Backup"


Verify started on 11/20/2012 at 9:40:44 AM.
Verify completed on 11/20/2012 at 9:40:44 AM.

Verify Set Summary
Verified 0 files in 16 directories.
Processed 7,788 bytes in  1 second.
Throughput rate: 0.446 MB/min


Set Information - sql01.ad.iss-eps.net  
Verify Set Information
Verify of "sql01.ad.iss-eps.net "
Backup set #5 on storage media #1
Backup set description: "sql Daily Incremental Backup"

Verify started on 11/20/2012 at 9:40:44 AM.

Verify Set Detail Information

Verify completed on 11/20/2012 at 9:43:07 AM.

Verify Set Summary
Verified 15 databases
Processed 4,216,934,830 bytes in  2 minutes and  23 seconds.
Throughput rate: 1687 MB/min



Job Completion Status
Job ended: Tuesday, November 20, 2012 at 9:43:07 AM
Completed status: Failed
Final error: 0xe000fec2 - A failure occurred accessing the SnapShot handle.
Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-65218

 

Exceptions
Click an exception below to locate it in the job log
Backup- sql01.ad.iss-eps.netV-79-57344-34100 - ADBO: Offhost backup initialization failure on: "\\sql01.ad.iss-eps.net\C:".

Snapshot provider error (0xE0008534): Backup Exec could not find a Microsoft Volume Shadow Copy Services (VSS) software or hardware provider on the media server. To use offhost backup, a VSS provider must be installed on both the media server and on the remote computer.

Check the Windows Event Viewer for details.


 

[marcusolini]

Hi Eddie.  Thank you for the debug information thus far.  In order to perform a complete analysis, the Remote Agent debug log from the affected Remote Server is needed.  Which from the provided Job Log appears to be Remote Server 'sql01.ad.iss-eps.net'.  When able please also provide the corresponding Remote Agent debug log for review.  Also, a quick question, is ADBO\Off-host being used for the backup?  Thank you.