Has anyone else run into this issue? VM backup fails with a "Status 13" or "Status 156" or "Status 40" - I have several hundred VMs and only a few end up in this state during a backup window. No snapshots show up in Snapshot Manager on the VMs that have this error. When consolidation is attempted VMware throws a "Unable to access file <unspecified filename> since it is locked" error. The "fix" for us has been to run "services.sh restart" on the ESXi server - this unlocks the file - then manually consolidating the VM. Sounds a lot like this issue: https://www-secure.symantec.com/connect/forums/problem-backup-virtual-machine-when-vm-convert-status-consolidate Master: 7.6.0.2 on Solaris 10 Media: 2.6.0.2 on 5220 Appliances

This appears to be similar as well: https://communities.vmware.com/message/2172747?tstart=2505

More interesting info. This is BackupExec-related: http://alpacapowered.wordpress.com/2012/06/04/snapshot-removal-issues-with-backupexec-and-locked-files/comment-page-1/

All backup appplication that use the VADP api's have orphaned/zombie snapshot issues at some time. We started using VADP backups early on and have found the orphaned snapshot problem is getting a lot less, The lessons we learnt was to: Keep NBU up to date as much as possible through patches and EBB's to fix VMware issues. Monitor all the VM machines for the presence of snapshots either from RVTOOLS (vHealth tab) or a script that lists all the deltas present in the environment. We decided against automated deletion as it was too risky. If you can't consolidate the snapshots try cloning the guest, cloning forces consildation. We successfully have done this when there were a very large number (hundrerds) of snapsots for a box. RVTOOLS is a good & free software for VMware reporting it will list other stuff like datastores and C: drives that are low of space etc if your software versions permit. Only requires a read-only VM account.

I run into this same thing occasionally. I do hotadd vm backups and sometimes the snapshots are still "hung-up" on the media server. I edit the media server vm and remove the "extra" disks....I can then run disk consolidation on the problem VM.

There is an Alarm definition in vSphere 5.x that could be used to setup email notification if you are wanting to closely monitor this.

VMware VM "needs consolidation" after backup failure

11 Replies

Replies have been turned off for this discussion

elanmbx
Level 6
11 years ago
This appears to be similar as well: https://communities.vmware.com/message/2172747?tstart=2505
elanmbx
Level 6
11 years ago
More interesting info. This is BackupExec-related: http://alpacapowered.wordpress.com/2012/06/04/snapshot-removal-issues-with-backupexec-and-locked-files/comment-page-1/
Jim-90
Level 6
11 years ago
All backup appplication that use the VADP api's have orphaned/zombie snapshot issues at some time. We started using VADP backups early on and have found the orphaned snapshot problem is getting a lot less,

The lessons we learnt was to:

Keep NBU up to date as much as possible through patches and EBB's to fix VMware issues.

Monitor all the VM machines for the presence of snapshots either from RVTOOLS (vHealth tab) or a script that lists all the deltas present in the environment.

We decided against automated deletion as it was too risky. If you can't consolidate the snapshots try cloning the guest, cloning forces consildation. We successfully have done this when there were a very large number (hundrerds) of snapsots for a box.

RVTOOLS is a good & free software for VMware reporting it will list other stuff like datastores and C: drives that are low of space etc if your software versions permit. Only requires a read-only VM account.
Bmitche
Level 5
11 years ago
I run into this same thing occasionally. I do hotadd vm backups and sometimes the snapshots are still "hung-up" on the media server. I edit the media server vm and remove the "extra" disks....I can then run disk consolidation on the problem VM.
Anonymous
11 years ago
There is an Alarm definition in vSphere 5.x that could be used to setup email notification if you are wanting to closely monitor this.
bcunha
Level 4
10 years ago
Seeing the same problem with a 5230 2.6.0.2 master/media (already present in 2.6.0.1) and VMware 5.5. I've already a case opened for a long time now, without any progress.
GuillaumeB
10 years ago
Hey!

I am experiencing the same issue, does anyone has an answer already?

Thansk!
VirtualED
Level 5
10 years ago
Please contact NetBackup support for a full investigation. These issues are solvable.

Also review late breaking news, there is a patch that is needed at 7.6.0.2 and 7.6.0.3 that may help.

http://www.symantec.com/docs/TECH199999

Beyond that, look out for running too many backups/snapshot operations at the same time. I/O contention is the major cause of consolidation failures.

Other issues I have found are:

1. Running multiple backups of the same VM at the same time.

2. Backups are hanging.
joyb81
Level 3
10 years ago
There are some methods which you may also do to fix it from vmware end http://aikitsupport.com/snapshot-consolidation-needed-fails-lock-message/

Let me know if any further need to lookup
ChristianWitzke
Level 3
10 years ago
SOLVED: with Information from the Website

1. migration to other Datastore

2. retry Consolidate