Forum Discussion

Bembel's avatar
Bembel
Level 3
12 years ago

Backup Exec hangs while backing up VMware VMs (snapshot)

We are having this issue for weeks/months now : while processing a backup-to-disk job using Backup Exec 2012 SP1a VMware infrastructure agent the backup process hangs at some point. We are running a VMware 5 (Build 768111) environment with a fibre channel SAN which is being backed up by a Windows 2008 R2 BE2012 SP1a (Verion 14.0 Rev. 1798 64 Bit) VM.

At some point in the backup process - and different VMs - I can see that there is no progress anymore for hours or days in my vSphere client (percentage of the job freezes) and in BE the MB/min indicator decreases and no changes to the number of backed up Bytes. This is only when using backup-to-disk, which actually is a (VMFS 5) partition within the BE virtual machine (drive letter V: size 1,5TB). Last time it stopped at 95% of our backend Exchange 2010 server (running, approx. 300GB), last time before it stopped while backing up a powered off Windows Server 2003 test system (approx. 30GB).

The only thing to do is hit cancel at the BE machine (which actually wouldn't cancel or stop the job), then reboot the BE machine, cancel the VMware job in vSphere client and then "delete" the VMware snapshot to return to merge the changes on the disks.

Gladly we back up all data (non-VMs) separately directly to tape (VMware Direct I/O connected SCSI LTO juke box), so I have at least my data at hand, but I don't understand why even a snapshot of a powered off system halts the Symantec VMware job. Snapshots manually initiated in vSphere client have not been an issue yet, disk space is available, too. Have not found any similar problem desciption in Google...

Any help really appreaciated!

2012
Agent for VMware Virtual Infrastructure
Backing Up
Backup and Recovery
Backup Exec
Troubleshooting
Virtualization
VMware
  • Bembel's avatar
    Bembel
    12 years ago

    Hi everyone,

    finally everything seems to be OK now. For me the solution was

    1.) uninstalling VMware VSS providers where BE VSS providers (agents) are installed,

    2.) deactivating GRT within these backups (for me not needed anyway) and

    3.) specifically choose Microsoft VSS provider under "Advanced Open File"

    For 3.) I got this hint from the Symantec support technician who helped me with another problem; he basically said that for VM AVVI backup taking snapshots under "Advanced Open File" must be activated, but the default to automatically chosse the snapshot provider is not the best setting because this could result in the problems described in this thread. The best choice is "System - Microsoft VSS provider".

    That was my choice yesterday and for the first time the job finished without any problem :-)

    Hope this helps!

15 Replies

Sort By
Show More
  • Bembel's avatar
    Bembel
    Level 3
    12 years ago

    Yesterday I have changed settings on my jobs and had to wait until the VMs rebooted on which I uninstalled VMware's VSS provider. Meanwhile I used the time to upgrade our vCenter infrastructure to the most current level (5.0 Update 1b, Build 804277) which today seems to have broken all my VM backup jobs - in VMware event viewer I can see that when triggering a backup job this message appears:

    Cannot login @<IP address of BE2012 server>

    On my BE log I get this error message : V-79-57344-38209

    Even after deleting the vCenter server in BE, adding it and successfully validating to every single VM and VM infrastructure folder the only machine I was able to backup is our VM template.

    While 5.0 Update 1 is supported in Symantec's BE2012 SCL (http://www.symantec.com/docs/TECH175581) most probably 5.0 Update 1b (released 2012-08-16) is not.

    Once I can back up our VMs again, I will test again.

  • unitofone's avatar
    unitofone
    Level 2
    12 years ago

    I'm having this exacty same issue as well.


    We are trying to backup two file servers, around 1.5TB in size as virtual machines to deduplicated storage with GRT.

    Physical Backup Exec server with SAN Transport enabled.

    Backup starts at around 3.5GB per minute but after about 15-20 minutes just seems to hang and the data rate doesn't change.
     

    There doesn't appear to be any logging of what is or isn't happening during this either, I will turn off AOFO and see if this helps.

     

     

  • PB_Fastener_IT's avatar
    PB_Fastener_IT
    Level 2
    12 years ago

    Hello Everyone.

    I spoke with a Symantec Tech as well as a VMWARE tech and they both stated the free esxi versions do not support vstorage API license which is needed in order for any vmware backup to work at all. If you have a standard, enterprise or enterprise plus vmware software then vstorage comes with it and you may have to confirm with symantec why it is not working, BUT if you do have the free ESXi versions then this is why no VMWARE backup works and just hangs or fails.

    regards,

    PB Fasteners IT Support

  • Bembel's avatar
    Bembel
    Level 3
    12 years ago

    Hi everyone,

    finally everything seems to be OK now. For me the solution was

    1.) uninstalling VMware VSS providers where BE VSS providers (agents) are installed,

    2.) deactivating GRT within these backups (for me not needed anyway) and

    3.) specifically choose Microsoft VSS provider under "Advanced Open File"

    For 3.) I got this hint from the Symantec support technician who helped me with another problem; he basically said that for VM AVVI backup taking snapshots under "Advanced Open File" must be activated, but the default to automatically chosse the snapshot provider is not the best setting because this could result in the problems described in this thread. The best choice is "System - Microsoft VSS provider".

    That was my choice yesterday and for the first time the job finished without any problem :-)

    Hope this helps!

  • JShetler's avatar
    JShetler
    12 years ago

    We had a similar issue with our virutal environment and backups.  Thanks to this thread we finally got some working backups, thanks!  However, we need to use GRT.  To fix the GRT issue we made three registry changes as directed by Symantec support:

    HKLM\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\VMware Agent

    Change "Disable NTFS Used Sector Tracking" from 0 to 1
    Change "Disable NTFS Used Sector Tracking Dynamic Disk Support" from 0 to 1
    Change "Disable NTFS Used Sector Tracking GPT Disk Support" from 0 to 1

    When we asked what exactly this does differently, all we were told was that it "disables features that are no longer needed."

    Afterwards our backups were able to run with GRT enabled, using AOFO and the Microsoft VSS provider.  Hopefully this will help save somebody the headache we've been dealing with.

    Edit:  Forgot to specify, registry changes were made to our vCenter/Backup Exec server.