04-17-2012 02:01 PM
For unknown reasons one particular VM will no longer do a weekly full backup (*.vmdk files) using the vSphere agent addon in Backup Exec 2010 R3.
Information About The VM:
Error:
Troubleshooting Steps I've Taken:
I'm kinda at my wits end troubleshooting this. Additional troubleshooting tips/suggestion great appreciated.
04-17-2012 02:22 PM
Is this a GRT or a non GRT backup failure ? Alternatively, when one fails, does the other complete ?
Have you customized the post thaw and the pre freeze scripts ?
Did you also uninstall VMware Filesystem Sync Driver from VMware tools ? (This would require a reboot)
04-17-2012 09:56 PM
You should push out the BE remote ageent again to the VM. This will disable the VMTools VSS provider and install the BE VSS provider which is the recommended provider.
04-18-2012 08:55 AM
Is this a GRT or a non GRT backup failure?
-I did see any specific mention to it being a GRT failure. Although the 'Use Backup Exec GRT' option is selected in the VMware vSphere agent backup job.
Alternatively, when one fails, does the other complete?
-The regular Windows server agent backup job does complete sucessfully (Incremental backup).
Have you customized the post thaw and the pre freeze scripts?
-No, I haven't customized any scrips. Honestly I don't even know what those are.
Did you also uninstall VMware Filesystem Sync Driver from VMware tools ? (This would require a reboot)
-I think so, I first:
04-18-2012 09:02 AM
Uncheck the usage of GRT from this VM's job & check if the backup completes successfully or not...
While installing VMWare tools, modify the install & Volume Shadow Copy Services Support along with VMware Filesystem Sync Driver can be uninstalled...
04-18-2012 10:46 AM
Are there any errors in vCenter when the snapshot attempt fails?
04-18-2012 12:21 PM
Uncheck the usage of GRT from this VM's job & check if the backup completes successfully or not...
-Unfortunately the backup job errored out again:
An attempt to take a snapshot of a VM failed because it was unable to quiesce an application.
While installing VMWare tools, modify the install & Volume Shadow Copy Services Support along with VMware Filesystem Sync Driver can be uninstalled...
-While doing this I noticed that Volume Shadow Copy Services Support is not installed. I was unable to find VMware Filesystem Sync Driver option in the VMware Tools Installation window.
VMware Tools Installation window (what I see):
Toolbox
VMware Device Driver
04-18-2012 01:00 PM
Oddly enough when I looked at the vCenter logs nothing
%ALLUSERSPROFILE%\VMware\VMware VirtualCenter\Logs\
However, when I left the vSphere client window open & ran the backup job I got an error:
The guest OS has reported an error during quiescing. The error code was:
5 The error message was: 'VmBackupOnThaw' operation failed.
3 The error message was: Custom quiesce script failed.
Never seen these errors before... good thing I left vSphere client running.
04-18-2012 01:12 PM
Is there a way to push out the agent again without uninstalling it from the server first?
04-18-2012 06:51 PM
Sure.
04-19-2012 01:47 PM
Based on the error I posted in reply to Ben. L's question I guess the problem is related with a customized the post thaw and the pre freeze scripts? Recommendations?
04-19-2012 06:29 PM
The following tricks working without uninstalling the VMWare tools support for VSS. Please be advised that I've tried this method on all of my failed to backup of Windows Server 2008/2003 x64 VMs
The reason why I'm doing this again is that I feel uncomfortable in removing the VSS snapshot provider support in VMWare tools:
1. Backup the following regkey: *HKLM\Software\Microsoft\EventSystem*
2. Erase the regkey HKLM\Software\Microsoft\EventSystem{26c409cc-ae86-11d1-b616-00805fc79216} an below
3. Reboot.
4. Verify that things are working by running
vssadmin list writers
vssadmin list providers.
and then in the services.msc list of services snap ins, make sure the following services are Started Automatically:
1. COM+ System Application
2. Distributed Transaction Coordinator
3. Volume Shadow Copy
4. VMware Physical Disk Helper Service
Automatically but not started:
1. Microsoft Software Shadow Copy Provider
2. VMware Snapshot Provider
This is the solution that i tried around myself without uninstalling the VSS Provider support in VMWare tools.
See the following KB URL: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1009073&sl...
Take a look at this TN also:
http://www.symantec.com/business/support/index?page=content&id=TECH70486
Hope this can be a helpful post for you.
05-03-2012 01:05 PM
Thanks for the reply. Once I deleted that registry key things (VSS & other services) went south. That's why we always backup.
05-03-2012 06:17 PM
So, is your problem solved?
05-17-2012 06:41 AM
Try to take some snapshot with quiesce mode enable at vcenter.
Remember to check the "quiesce mode" check box.
Maybe the root cause is the VMware.