NetBackup 7.5 - VMware Backup - Snapshot Jobs stay active for long time even snapshot is already removed

[julmer]

Hi,

we are using NetBackup 7.5.0.3 to Backup a VMware vCloud Infrastructure.
Backup works fine but there is a very long delay (at least 10min) between the point were the snapshot removal tasks completes (in vCenter) and the snapshot job finishes in the NBU activity monitor.

As VMware vCloud can deploy linked clones simulataneous snapshot operations can cause many IO cmds on the same VMDK. Therefore we set resource limits and restricted the number of parallel snapshot operations.

We set the "disableRestoreInfo" as recommended in http://www.symantec.com/business/support/index?page=content&id=TECH129568 but I don't know if this setting is also for NBU 7.5. Can't say if this has any effect at this time.

Is this a known issue? Any hints?

NBU is 7.5.0.3 with latest EEB
Windows 2008 R2
VMware vSphere & vCenter U1
VMware vCloud Director 1.5.1

Kind regards
Julian

[julmer]

We figured out that the long delay does not rely on the vCloud Director or Linked Clone Environment.
The same behavior can be observed in a normal vCenter with Flat VMDKs.

[blakberye]

With 7.5, the snapshot and the backup are separate jobs.  Previously, the snapshot was created in the parent job, with the backup being a child.  Now, they are treated as two separate jobs.  So the snapshot is created, and the backup then runs, but is queued due to resource limits being hit. 
If policies use query builder, instead of browse, set the Max Simultaneous Snapshots per vCenter from unlimited (default) to 5-10. This will only throttle the number of snaphot related activities, but will allow your backups to run per your other settings.

http://www.symantec.com/business/support/index?page=content&pmv=print&impressions=&viewlocale=&id=HO...

[julmer]

Hi blakberye,

we already set the resource limits. We only want 2 simultaneous snapshot activities as we use linked clones. When a VM has a greater chain length removing snapshots becomes very IO intesive.

This also does not solve the problem nor explain why it takes at least 10mins after a snapshot job is finished in NBU while the snapshot task is finished long time ago in vCenter.

If you have many VMs your backup windows explodes although the backup itself takes 20% of the time.
I think this is an API or trigger related bug/problem from NBU.

Kind regards
Julian

[ALE_DA_SUL]

Hi Julmer,

I was with the same problem, take a look in this technote:

http://www.symantec.com/business/support/index?page=content&id=TECH161502

Kind regards,

Alexandre Gomes

[D_Thomas]

I am having the same problem.

Netbackup 7.5.0.4

Windows 2008 R2 Media Server

Some snapshot processes take a whole hour for very small Virtual Machines. Some take no time at all.

The VMware snapshot processes (create and delete) run just fine.

Its not causing problems yet, but I am getting some 196 here and there because the window is too small. I opened it up a bit, but this doesn't make a whole let of sense to me.

DT

[D_Thomas]

So I have found a work around for this. I noticed that the media server had an inordinate amount of bpfis processes running. One of the parameters in the Resource Limits options in the master server is to limit the amount of snapshot processes, and I dropped this to 12. Its seemed to make a huge impact on getting the snapshots done more quickly.

I could probably increase this number, but so far this is working well for my environment.

DT