VMWare Backup with Netbackup failure with status 156

[maxsven]

I've created a policy to backup 3 VMs using "VMWare" backup policy, the 3 Ms are turned off... only one client that i havent been able to backup... I keep getting status 156

Can anyone please help?

Netbackup 7.5

VSphere 4.1

Logs:

all resource requested successfully:

30/11/2012 5:05:10 PM - started
30/11/2012 5:05:11 PM - estimated 0 Kbytes needed
30/11/2012 5:05:11 PM - begin Parent Job
30/11/2012 5:05:11 PM - begin VMware, Start Notify Script
30/11/2012 5:05:13 PM - Info RUNCMD(pid=7604) started           
30/11/2012 5:05:13 PM - Info RUNCMD(pid=7604) exiting with status: 0        
Status 0
30/11/2012 5:05:13 PM - end VMware, Start Notify Script; elapsed time: 00:00:02
30/11/2012 5:05:13 PM - begin VMware, Step By Condition
Status 0
30/11/2012 5:05:13 PM - end VMware, Step By Condition; elapsed time: 00:00:00
30/11/2012 5:05:13 PM - begin VMware, Read File List
Status 0
30/11/2012 5:05:13 PM - end VMware, Read File List; elapsed time: 00:00:00
30/11/2012 5:05:13 PM - begin VMware, Create Snapshot
30/11/2012 5:05:16 PM - started process bpbrm (12352)
30/11/2012 5:05:18 PM - Info bpbrm(pid=12352) ******* is the host to backup data from    
30/11/2012 5:05:18 PM - Info bpbrm(pid=12352) reading file list from client       
30/11/2012 5:05:18 PM - Info bpbrm(pid=12352) start bpfis on client        
30/11/2012 5:05:18 PM - Info bpbrm(pid=12352) Starting create snapshot processing        
30/11/2012 5:05:20 PM - Info bpfis(pid=13936) Backup started          
30/11/2012 5:05:23 PM - snapshot backup of client ****** using method VMware_v2
30/11/2012 5:05:39 PM - Critical bpbrm(pid=12352) from client ********: FTL - snapshot creation failed, status 156  
30/11/2012 5:05:40 PM - Warning bpbrm(pid=12352) from client ******: WRN - ALL_LOCAL_DRIVES is not frozen   
30/11/2012 5:05:41 PM - end writing
Status 156
30/11/2012 5:05:41 PM - end VMware, Create Snapshot; elapsed time: 00:00:28
30/11/2012 5:05:41 PM - begin VMware, Stop On Error
Status 0
30/11/2012 5:05:41 PM - end VMware, Stop On Error; elapsed time: 00:00:00
30/11/2012 5:05:41 PM - begin VMware, Delete Snapshot
30/11/2012 5:05:44 PM - started process bpbrm (8216)
30/11/2012 5:05:47 PM - Info bpbrm(pid=8216) Starting delete snapshot processing        
30/11/2012 5:05:47 PM - Info bpfis(pid=0) Snapshot will not be deleted       
30/11/2012 5:05:52 PM - Info bpfis(pid=6328) Backup started          
30/11/2012 5:05:52 PM - Critical bpbrm(pid=8216) from client ******: cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.*****_1354255510.1.0     
30/11/2012 5:05:53 PM - end writing
Status 1542
30/11/2012 5:05:53 PM - end VMware, Delete Snapshot; elapsed time: 00:00:12
Status 156
30/11/2012 5:05:53 PM - end Parent Job; elapsed time: 00:00:42
snapshot error encountered(156)

[Yasuhisa_Ishika]

Please check bpfis debug log. Any relevant messages will be logged in it.
Debug logging is disabled by default. Before you reproduce this issue, you need to create bpfis folder in install_path\Netbackup\logs to enable it.
Also check if you can manually create snapshot of this VM from vCenter.

[Marianne]

Have you verified all pre-reqs in NBU for VMware Admin Guide such as vmtools installed, no RDM dksk, etc, etc... Can you take a snapshot of this VM from within VMware? Have you checked for left-over snapshots? Have you checked the Troubleshooting section of the manual? Do you have bpfis log enabled on the backup host?

[maxsven]

Hi ,

I do have debug enabled and have looked at the logs in the bpfis folder... nothing relvant that i can see there...

Yes i am able to take a snapshot manually in vmware...

anything else you can think of?

[maxsven]

yes no RDM and VMWare tools not relevant as the VM is turned off.

a manual snapshot works fine, all snapshots consolidated.

yes for the bpfis logs but i really can't see much relevant in the log file to this backup job.

[RLeon]

As Marianne pointed out, there is an entire troubleshooting section in the Nbu VMware admin guide about "Snapshot error encountered (status code 156)":

If the virtual machine is powered off:
- If NetBackup uses credentials for an ESX server instead of vCenter, it may not be able to identify a powered off virtual machine.
- If the policy uses VM hostname or VM DNS name as the Primary VM identifier, NetBackup may not find the virtual machine. The backup fails.
- If the policy uses VM display name or VM UUID as the Primary VM identifier, NetBackup can identify the virtual machine. The backup succeeds.

So I guess try using vCenter credential instead of ESX credential, and use VM display name/UUID instead of the VM's network name.

[maxsven]

Hi,

Yes I've seen this, I'm using vcenter credentials and the VM display name... still to no avail...

thanks.

[maxsven]

When I try to validate the backup policy, it mainly complains about "snapshot backup of client using method VMware_v2"