Netbackup 7.5.0.5 not releasing Virtual Drives

[evargas]

Hi all,

This is the issue:

Netbackup servers 7.5.0.5 (Linux RH)

Netbackup backup hosts is a virtual Machine (Windows 2008 R2) running NBU 7.5.0.5 Client

Backups start and the VMDrives are mounted in the backup host (using hotadd), backup complete but all the Virtual Drives still mounted in the backup host.

the next day is the same issue, this is creating many snap shots of these VM in the storage.

Is there a solution to this?

We have opened a case with Symantec support but they can't find anything wrong in the log files.

any help will be appreciated.

Thanks

[Will_Restore]

One possibility below.  If that doesn't help I would contact VMWare Support.

http://www.vmware.com/support/developer/vddk/VDDK-1.2.1-Relnotes.html

• Mounting HotAdded disks may fail and not be cleaned up

  In some cases, an attempt to mount a HotAdded disk fails. In such cases, directories can be created but are not removed by the standard cleanup process. To resolve this issue, manually remove the disks from the proxy virtual machine and delete the \tmp\vmware directory created in the current user's Documents and Settings directory, typically found at C:\Documents and Settings\username\Local Settings\Temp\vmware-username\. The next time a snapshot is created or deleted for the virtual machine, redo logs will get cleaned up.

[evargas]

Unfotunatelly that did not apply to my case, that directory/file doesn't exist in my WIndows 2008 backup host.

we have a case with VMWare but they are pointing a finger to Symantec, saying that Netbackup has a lock on the snapshot and they can't delete them.

[sri_vani]

• During a virtual machine backup, VMware creates mount directories in the C:\Windows\Temp folder on the backup host. After the backup, VMware automatically removes the mount directories. In some cases, such as when the backup uses the hotadd transfer type, NetBackup may be unable to remove the mount directories.

  Remove the mount directories from the \Temp folder and retry the backup. The folder name contains a string such as VMware-System.

[evargas]

Thanks sri vani

unfortunatelly this did not resolved the issue.

we are working with Vmware and Symantec to find a resolution, we have stopped all backups bacause this is causing a lot critical issues in the Vsphera storage.

[sri_vani]

Evargas, Plz do share with us once it gt fixed

[RamNagalla]

hi,

what the Vcenter version?

does the VM using the application-based snapshots or file system quiesced snapshots.

if it using the  application-based snapshots, try to disable it and use the file system quiesced snapshots.

this modification requires to change in VMX file and need a reboot of the server...

[evargas]

We still working on this, VMware suggested the below resolution wich we are doing at this time.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2017072

Nagalla,

vcenter version is 4.1.0

We are not using Application-base snapshoots

[RamNagalla]

i was just going thourgh this doc and thoght of sharing.. please check if your setup is honored all the below requirements and limitations

Page number 35 from the netbackup VMware admin guide.

Notes on the hotadd transport mode

NetBackup supports several transport modes for sending snapshot data between

the VMware datastore and the VMware backup host during a backup or restore.

One of those transport modes (hotadd) is used when the VMware backup host is

installed in a virtual machine.

Note the following about the hotadd transport mode:

■ For a backup or restore, the hotadd transport mode does not support the vmdk

files that are larger than 1 TB. If the .vmdk file is larger than 1 TB, the file cannot be mounted.

■ The VMware backup host must be installed in a virtual machine.

NetBackup character restrictions for virtual machine display names

■ For ESX versions 3.5 and earlier: You must configure a shadow (helper) virtual

machine before the backup starts. The helper virtual machine must have at

least as many SCSI controllers as exist on the virtual machine that you want

to back up.

The helper virtual machine is used internally by the VMware VDDK to attach

the disks temporarily during hotadd backup or restore.

Note: A helper virtual machine is not required for backups of ESX 4.0 (or later)

virtual machines. For earlier ESX versions, a helper virtual machine is required.

See “Creating a helper virtual machine for hotadd transport” on page 37.

■ The following is a VMware requirement: The virtual machine to back up and

the virtual machine that contains the hotadd backup host must reside in the

same VMware datacenter. The same VMware requirement applies to virtual

machine restore: The virtual machine to restore and the virtual machine that

contains the hotadd restore host must reside in the same VMware datacenter.

For hotadd backup, Symantec recommends at least one hotadd backup host

for each datacenter.

■ NetBackup does not support IDE disks on the virtual machine.

■ On the virtual machine to back up, no two disks should have the same name.

(Identical names can occur if the disks reside on different datastores.)

■ The ESX server (where the backup-host virtual machine resides) must have

access to the datastore of the virtual machines that you want to back up.

■ The datastore for the backup-host virtual machine must have some free space

before the hotadd backup begins. Otherwise, the backup may fail.

■ For additional hotadd information, see the following VMware document:

VDDK 5.0 Release Notes

■ The Enable file recovery fromVMbackup option does not work with the

hotadd transport mode in the following case: The virtual machine that contains

the backup host is a clone of the virtual machine that is backed up (or vice

versa). In this case, a backup that uses the hotadd transfer mode may appear

to succeed. However, some of the virtual machine volumes may be incorrectly

mapped and cannot be restored.

The converse is also true. The virtual machine that you back up with the hotadd

transport mode cannot be a clone of the virtual machine that contains the

backup host. Furthermore, the two virtual machines must not be clones of the

same parent.

VMware has identified this issue as software defect SR#1442259141. To use

the hotadd transport mode with your backup, do the following: Make sure the

Notes and prerequisites

virtual machine that contains the VMware backup host and the virtual machine

you back up are not clones of each other.

[evargas]

Nagalla,

our VMWare setup complies with all the requirements, we have been successfully backing up this environment for a while.  we are investigating what changed a month ago which is when we started having this issue.