10-30-2012 02:22 PM
Hello Techies,
Need help in fixing the issue, Error - (156) snapshot error encountered.
Error occurred while performing backup of a VM machine.
This error occurred at 0:02:15 after the backup is triggered. Pasted the detailed status below, The NBU Media and master server are different, we have Windows 2003 R2 as the media server OS and its x86 bit OS and Master Server is IBM AIX 5.3.
This is reoccurring issue, the backup is failing often. There are no environmental changes carried out recently on both the NBU and client end.
The Data Store where this VM resides is on a XIV storage box and the ESX version is 4.
Let me know how to permanently fix this snapshot error issue or need a workaround if the final solution indicates to up gradation.
Let me know what more details required to fix this EC.
Detailed Job Status:
10/31/2012 00:49:38 - requesting resource xx-xxxx-xxx04-b-hcart3-robot-tld-0
10/31/2012 00:49:38 - requesting resource XX-XXXX-XXXX01.NBU_CLIENT.MAXJOBS.<Client Server Name>
10/31/2012 00:49:38 - requesting resource XX-XXXX-XXXX01.NBU_POLICY.MAXJOBS.VMWARE3-VCB
10/31/2012 00:49:38 - Info nbrb (pid=2162758) Limit has been reached for the logical resource XX-XXXX-XXXX01.NBU_POLICY.MAXJOBS.VMWARE3-VCB_VCB
10/31/2012 00:50:04 - begin Create Snapshot
10/31/2012 00:50:04 - snapshot backup of client <Client Server Name> using method VMware
10/31/2012 00:49:59 - Waiting for scan drive stop IBM.ULT3580-TD3.018, Media server: xx-xxxx-xxx04-b
10/31/2012 00:50:00 - granted resource XX-XXXX-XXXX01.NBU_CLIENT.MAXJOBS.<Client Server Name>
10/31/2012 00:50:00 - granted resource XX-XXXX-XXXX01.NBU_POLICY.MAXJOBS.VMWARE3-VCB
10/31/2012 00:50:00 - granted resource 800921
10/31/2012 00:50:00 - granted resource IBM.ULT3580-TD3.018
10/31/2012 00:50:00 - granted resource xx-xxxx-xxx04-b-hcart3-robot-tld-0
10/31/2012 00:50:01 - estimated 1132190 kbytes needed
10/31/2012 00:50:01 - begin Parent Job
10/31/2012 00:50:01 - begin Flash Backup Windows: Start Notify Script
10/31/2012 00:50:02 - started process RUNCMD (pid=1163408)
Operation Status: 0
10/31/2012 00:50:02 - end Flash Backup Windows: Start Notify Script; elapsed time 0:00:01
10/31/2012 00:50:02 - begin Flash Backup Windows: Step By Condition
Operation Status: 0
10/31/2012 00:50:02 - end Flash Backup Windows: Step By Condition; elapsed time 0:00:00
10/31/2012 00:50:02 - begin Flash Backup Windows: Read File List
Operation Status: 0
10/31/2012 00:50:02 - end Flash Backup Windows: Read File List; elapsed time 0:00:00
10/31/2012 00:50:02 - begin Flash Backup Windows: Create Snapshot
10/31/2012 00:50:03 - started process bpbrm (pid=14468)
10/31/2012 00:51:53 - Critical bpbrm (pid=14468) from client <Client Server Name>: FTL - snapshot creation failed, status 156
10/31/2012 00:51:54 - end Flash Backup Windows: Create Snapshot; elapsed time 0:01:52
10/31/2012 00:51:54 - end writing
Operation Status: 156
10/31/2012 00:51:54 - end Parent Job; elapsed time 0:01:53
10/31/2012 00:51:54 - begin Flash Backup Windows: Stop On Error
Operation Status: 0
10/31/2012 00:51:54 - end Flash Backup Windows: Stop On Error; elapsed time 0:00:00
10/31/2012 00:51:54 - begin Flash Backup Windows: Delete Snapshot On Exit
10/31/2012 00:51:54 - begin Flash Backup Windows: Delete Snapshot On Exit
10/31/2012 00:51:56 - started process bpbrm (pid=11744)
10/31/2012 00:51:57 - end writing
Operation Status: 0
10/31/2012 00:51:57 - end Flash Backup Windows: Delete Snapshot On Exit; elapsed time 0:00:03
10/31/2012 00:51:57 - begin Flash Backup Windows: End Notify Script
10/31/2012 00:51:58 - started process RUNCMD (pid=917528)
Operation Status: 0
10/31/2012 00:51:58 - end Flash Backup Windows: End Notify Script; elapsed time 0:00:01
Operation Status: 156
10/31/2012 00:51:58 - end Flash Backup Windows: Delete Snapshot On Exit; elapsed time 0:00:04
snapshot error encountered (156)
Solved! Go to Solution.
12-23-2012 02:15 AM
Hello Friends,
Apologies for delay in replying to this thread, as I was busy in few reporting tasks.
Thanks for your help on this part to fix this 156 error code. Appreciate every one for providing your valuable information.
But the issue was persisting due to LUN visibility, The ESX box where unable to see the newly assigned LUN, Because of this the backup job failed.
We have restarted the VCBmounter daemon on respective server and re-scan all the LUNs with the help of storage and VM team, Now the backup is running fine and the issue was fixed.
Once again thanks for your help guys.
10-30-2012 03:01 PM
Snapshot creation failed. Can you create the snapshot on the VM?
Do you have the bpfis log from the backup host? post that up?
10-30-2012 07:38 PM
I agree, in my environment when I see this I look to my vCenter server and what is logged there during the snapshot creation. Are there any errors there? Try and manually create the snapshot from vCenter or the VM.
Are other similar backups working for the same vCenter/ESXserver working?
If you see nothing in vCenter start looking in the bpfis logs.
11-02-2012 05:01 PM
I had requested VM team to provide a temp access for me to the Vcenter in which the client resides.
Once i had the access i will look into the manuall creation on snapshot.
11-09-2012 05:33 AM
Often find that snapshot errors on windows systems is related to lack of space in shadow storage area and/or disk in the VM.
The application and system event log the client can often give hints to what problem are.
12-08-2012 02:51 AM
I agreed , please check daatstore space as well as client vm (Guest Vm ) OS drive space. For Snapshot backups, You should have minimum space in Datastore & guest vm as well.
Regards,
Sathish
12-23-2012 02:15 AM
Hello Friends,
Apologies for delay in replying to this thread, as I was busy in few reporting tasks.
Thanks for your help on this part to fix this 156 error code. Appreciate every one for providing your valuable information.
But the issue was persisting due to LUN visibility, The ESX box where unable to see the newly assigned LUN, Because of this the backup job failed.
We have restarted the VCBmounter daemon on respective server and re-scan all the LUNs with the help of storage and VM team, Now the backup is running fine and the issue was fixed.
Once again thanks for your help guys.