04-26-2017 06:43 AM
Hello,
Netbackup Environment on 7.6.1
Master on Linux 6.2
Media Server **bleep** Backup Host as Netbackup Appliance 5230 on 2.6.1
Linux VM on RHEL 6.2 with 12 TB of Disk size.
VMware policy
Backups are failing since few days after the filesystem extension was done on a Logical Volume, the backup works without File Level Recovery option but fails with error 50 when FLR is enabled and complains about:
GetDiskChangedInfoAPI: SYM_VMC_ERROR: SOAP_ERROR
04/15/2017 18:25:52 : g_vmmInterfaceLogger:libvix.cpp:1890 <INFO> : SOAP 1.1 fault: "":ServerFaultCode [no subcode]
Any help on this topic is much appreciated.
Solved! Go to Solution.
04-26-2017 07:21 AM
Hi,
Did you check the Netbackup EEB Guide ? Maybe it can be a problem with NBU version 7.6.1. I was seeing the Netbackup EEB Guide Release 7.6 and 7.6.x.x ( Version 8), and I found these EEBs for this version.
Have you applied them in your environment?
Look:
Etrack Incident: 2990696
■ Associated Primary Etrack: N/A
■ Associated Service Request Etrack(s): N/A
■ Description:
An issue occurred where VMware backups would fail with status codes 50 and
84 and generate a backup-and-archive manager (bpbkar) core dump issue on
a NetBackup appliance.
EEB 3643984
Hot fix or EEB Identifier
3643984
Description - Logical volume mapping (LVM) fails due to an incorrect offset for one of the file extents during VMware backup.
Resolved in Versions - 8.0, 7.7.3, 7.7.2, 7.7.1, 7.7, 7.6.1.2
04-26-2017 06:46 AM
04-26-2017 07:18 AM
What is your ESXi version?
What was the original disk space before extending it?
I assume when you say the CBT reset option, you are referring to either doing this through vSphere or using powerCLI is that correct? Simply disabling BLIB in the backup policy does not turn off CBT.
04-26-2017 07:21 AM
Have you checked this document to see if this VM is supported for File Recovery option?
Statement of Support for NetBackup 7.x and 8.x in a Virtual Environment: http://www.veritas.com/docs/000006177
See table 12 : VMware guest operating systems supported for file-level recovery
04-26-2017 07:21 AM
Hi,
Did you check the Netbackup EEB Guide ? Maybe it can be a problem with NBU version 7.6.1. I was seeing the Netbackup EEB Guide Release 7.6 and 7.6.x.x ( Version 8), and I found these EEBs for this version.
Have you applied them in your environment?
Look:
Etrack Incident: 2990696
■ Associated Primary Etrack: N/A
■ Associated Service Request Etrack(s): N/A
■ Description:
An issue occurred where VMware backups would fail with status codes 50 and
84 and generate a backup-and-archive manager (bpbkar) core dump issue on
a NetBackup appliance.
EEB 3643984
Hot fix or EEB Identifier
3643984
Description - Logical volume mapping (LVM) fails due to an incorrect offset for one of the file extents during VMware backup.
Resolved in Versions - 8.0, 7.7.3, 7.7.2, 7.7.1, 7.7, 7.6.1.2
04-27-2017 12:56 AM
ESX Version is 5.5 U3
CBT reset was done from vsphere and even I have disabled all Optimized options, ran a full backup and turned them back ON.
04-27-2017 01:01 AM
Thankyou Marianne,
Yes it is in the supported list, RHEL 6.2 and NBU installed is 7.6.1 on Master and Appliance on 2.6.1.
The problem here seems to be that the Netbackup is unable to read information about modified time stamps etc and even CBT enabled backup is happening Full everyday with below statement in logs:
04/26/2017 10:32:37 - Info bpbrm (pid=14022) reading file list for client
04/26/2017 10:32:37 - Info bpbrm (pid=14022) accelerator enabled
04/26/2017 10:32:37 - Info bpbrm (pid=14022) There is no complete backup image match with track journal, a regular full backup will be performed.
04/26/2017 10:32:37 - Info bpbrm (pid=14022) starting bpbkar on client
04-27-2017 01:14 AM
Thanks Thiago,
Let me check, this information can turn out to be very helpful.
04-27-2017 01:24 AM
Can someone please share the link to download the EEB particularly for 7.6.1 Netbackup Environment and any document detailing about where to install (on server, backup host (Appliance) etc ....
04-27-2017 01:31 AM
So you followed these instructions on how to reset the CBT?
Please detail the steps you took to reset the CBT.
You didn't mention the original FS size and how much it was extended by.
04-27-2017 02:59 AM
Yes Kruton,
I have used the mentioned VMware article to reset the CBT.
all ctkEnabled parameters were set to False and then backup ran without CBT enabled (all optimization options turned OFF) and then turning ON the optimization options (keeping File Level Recovery turned OFF) and then trigger another backup.
Full Backup happens successfully but as soon as you turn ON the File Level Recovery option then it starts failing.
Filesystem size was 10.5 TB earlier and extended to 14 TB.
04-27-2017 03:11 AM - edited 04-27-2017 03:11 AM
What FS type is being backed? You are aware not all FSes are allowing the file level recovery...
Please this exceprt from NetBackup for VMware Compatibility
"Support for the GUID Partition Table (GPT) starts at NetBackup 7.6.0.2.
Note: Granular recovery (GRT) of Exchange or SharePoint data that resides on a GPT disk is not supported from a backup image created for a VMware backup.
LVM2 volume manager is also supported. Support for ext4 starts at RHEL 5.4 and SUSE 11.
Support for file-level recovery for ext4 64-bit starts at NetBackup 8.0, at RHEL 7 and SUSE 12."
04-27-2017 03:18 AM
Hello Quebek,
This setup was working with File Level Recovery option enabled until the File System was extended (co-incidence may be). The FS type is ext4 32 bit and is supported.
04-27-2017 03:50 AM
Well then I would suggest to open a case with vrts...
04-27-2017 06:32 AM
Hi dipendra, I was looking for something at VMware side, and I found something in VDDK 5.5 Release Notes.
See if this applies for you case.
VDDK 5.5 Release Notes - https://www.vmware.com/support/developer/vddk/vddk-550-releasenotes.html
QueryChangedDiskAreas API returns incorrect sectors after extending virtual machine VMDK file with Changed Block Tracking (CBT) enabled (2090639) - https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=20906...
Thiago.
04-28-2017 03:01 AM
Hi,
NBU version, OS Version, Vmware version are not sufficient. Please Checl all notes about partitions, LVM2 and Filesystems. maybe you are run out of support by changig your LVM-layout.
ciao
tunix2k
04-28-2017 03:45 AM
Thanks Tunix2k,
Please elaborate by an example, where should I look at exactly ? Any commands to check on VM ?
04-28-2017 05:20 AM
Hi,
I have checked it. There is nothing. @quebek are gave you the right informations. Sorry.
Since CBT is checked I would open a case like @quebek suggested.
On other hand 7.6.x is no longer supportet. And @Thiago_Ribeiro showed an EEB 3643984 which is fixed in 7.7.
What about an upgrade?
tunix2k
04-30-2017 05:05 AM
Thanks tunix2k,
Yes I am planning an upgrade but there are certain application dependencies hence is the delay.
Secondly I wanted to test the EEB in my lab with 7.6.1.2 installed on a VM server, however I cannot see the credentials tab to add Virtual Center Server in the Netbackup console. Any ideas around this ?