Windows 2008 Vmware backup issues

[AlGon]

Netbackup 7.0 – Window 2003 Server

VMware 4.1 - Netbackup backing up Vms using Vstorage API

We’re having a few issues with our new VMware Windows 2008 guests locking up occasionally during the snapshot process of the backup.

 The system event log shows the error

Event ID 12

Source PlugPlayManager

 The device 'VMware Virtual disk SCSI Disk Device' (SCSI\Disk&Ven_VMware&Prod_Virtual_disk\5&22be343f&0&000300) disappeared from the system without first being prepared for removal.

 This says to me that maybe the OS isn’t being quiesed when the snapshot is taken. 

 Has anyone else had this problem, and is there a fix or workaround?

 Cheers

[rizwan84tx]

Please post the events reported in vCenter during the snapshot initiation, perhaps that could give a clue. Also check if the LUN volume allocated for this VM is visible to the ESX host.

[AlGon]

The events just show that the snapshot is created successfully followed by two disk releases.  No error events are displayed within VCenter.

[AlGon]

The events just show that the snapshot is created successfully followed by two disk releases.  No error events are displayed within VCenter.

[rizwan84tx]

How many vmdk are there for this VM? Also check if the Datastore LUN for this VM is presented properly to backup host and visible to ESX.Please check if the VM has any snapshots in the Snapshot manager.

Please attach the bpfis log and activity monitor detailed status information.

[AlGon]

Hi Rizwan

There is only one VMDK attached to this server.  I've checked the Datastore LUN is presented properly and it is.

01/06/2011 12:04:02 - requesting resource VTL-StorageUnit
01/06/2011 12:04:02 - requesting resource MasterServer.NBU_CLIENT.MAXJOBS.VMguest.domain.uk
01/06/2011 12:04:02 - requesting resource MasterServer.NBU_POLICY.MAXJOBS.VMware_Policy
01/06/2011 12:04:03 - granted resource MasterServer.NBU_CLIENT.MAXJOBS.VMguest.domain.uk
01/06/2011 12:04:03 - granted resource MasterServer.NBU_POLICY.MAXJOBS.VMware_Policy
01/06/2011 12:04:03 - granted resource Tape1
01/06/2011 12:04:03 - granted resource VTL-Drive
01/06/2011 12:04:03 - granted resource VTL-StorageUnit
01/06/2011 12:04:03 - estimated 853222 Kbytes needed
01/06/2011 12:04:03 - begin Parent Job
01/06/2011 12:04:03 - begin Flash Backup Windows, Start Notify Script
01/06/2011 12:04:03 - started process RUNCMD (7568)
01/06/2011 12:04:03 - ended process 0 (7568)
Status 0
01/06/2011 12:04:03 - end Flash Backup Windows, Start Notify Script; elapsed time: 00:00:00
01/06/2011 12:04:03 - begin Flash Backup Windows, Step By Condition
Status 0
01/06/2011 12:04:03 - end Flash Backup Windows, Step By Condition; elapsed time: 00:00:00
01/06/2011 12:04:03 - begin Flash Backup Windows, Read File List
Status 0
01/06/2011 12:04:03 - end Flash Backup Windows, Read File List; elapsed time: 00:00:00
01/06/2011 12:04:03 - begin Flash Backup Windows, Create Snapshot
01/06/2011 12:04:03 - started
01/06/2011 12:04:04 - started process bpbrm (5244)
01/06/2011 12:04:05 - begin Create Snapshot
01/06/2011 12:04:09 - snapshot backup of client VMguest.domain.uk using method VMware
01/06/2011 12:05:24 - end Create Snapshot; elapsed time: 00:01:19
01/06/2011 12:05:24 - end writing
Status 0
01/06/2011 12:05:24 - end Flash Backup Windows, Create Snapshot; elapsed time: 00:01:21
01/06/2011 12:05:24 - begin Flash Backup Windows, Policy Execution Manager Preprocessed
Status 0
01/06/2011 12:08:42 - end Flash Backup Windows, Policy Execution Manager Preprocessed; elapsed time: 00:03:18
01/06/2011 12:08:42 - begin Flash Backup Windows, Validate Image
Status 0
01/06/2011 12:08:42 - end Flash Backup Windows, Validate Image; elapsed time: 00:00:00
01/06/2011 12:08:42 - begin Flash Backup Windows, Delete Snapshot
01/06/2011 12:08:44 - started process bpbrm (6328)
01/06/2011 12:08:59 - end writing
Status 0
01/06/2011 12:08:59 - end Flash Backup Windows, Delete Snapshot; elapsed time: 00:00:17
01/06/2011 12:08:59 - begin Flash Backup Windows, Delete Snapshot On Exit
Status 0
01/06/2011 12:08:59 - end Flash Backup Windows, Delete Snapshot On Exit; elapsed time: 00:00:00
01/06/2011 12:08:59 - begin Flash Backup Windows, End Notify Script
01/06/2011 12:08:59 - started process RUNCMD (5576)
01/06/2011 12:08:59 - ended process 0 (5576)
Status 0
01/06/2011 12:08:59 - end Flash Backup Windows, End Notify Script; elapsed time: 00:00:00
Status 0
01/06/2011 12:08:59 - end Parent Job; elapsed time: 00:04:56
the requested operation was successfully completed(0)

[rizwan84tx]

Does the VM Guest has any Volume Shadow copies? if it has any copies please delete them. OS fails while trying to Quiesce the shadow copy thinking as a Disk and reports the event 12.

[emret]

VSS option for this client ?

[AlGon]

Hi.

No, it doesn't currently have any shadow copies in place.  VSS is enabled.

[rizwan84tx]

Any Error codes reported for the failure in NBU? How fast does the snapshot is deleted since the time of creation?

[AlGon]

Hi Rizwan,

NBU doesn't report any errors.  The only error reported is the one in windows which I stated above.

Event ID 12

Source PlugPlayManager

 The device 'VMware Virtual disk SCSI Disk Device' (SCSI\Disk&Ven_VMware&Prod_Virtual_disk\5&22be343f&0&000300) disappeared from the system without first being prepared for removal. 

[rizwan84tx]

Please check this microsoft link http://support.microsoft.com/kb/958669 (win 2008)

http://support.microsoft.com/kb/828285  (win 2003)

Microsoft states that this error can be safely ignored if there are no hardware volume shadow copies. And, its the problem with the OS.