cancel
Showing results for 
Search instead for 
Did you mean: 

vCenter Appliance Upgrade to 6.0u1b / Resource Contention

ianhoskins
Level 4

Wanted to get some input from folks here.
NBU 7.6.1.2 on Linux (Master and Media).  VMWare backup running like a champ for about 1.5 years.  VMWare vCetner Appliance (VCSA) was upgrades to 6.0u1b (was 6.0).  After that upgrade we started recieving multiple backup failures.

NBU Jave Console Shows the following
SnapShot shows an error 6 (Snapshot creation and deletion in VMware is successful)
Backup shows an error of 23 (Error opening snapshot disks) and 6 (cleanup failed, failed to backup requestd files)

The Job can be restarted and is successful, so I know its not a LUN mapping issues or credential issue.
Other jobs that are started at the exact some time (+/- a few seconds) and on the same datastore and esx server will run without issues.

The VXMS logs that I have collected have not shown any issues according to Veritas.  The only thing they have mentioned is a insuffucient buffer size, but they don't seem concerned about that.
 

   01/28/2016 19:00:06 : g_vixInterfaceLogger:libvix.cpp:1775 <DEBUG> : [VFM_ESINFO] VixDiskLib: VixDiskLib_GetMetadataKeys: Insufficient buffer size (has 0, need 202): Error 24 at 4519.

   01/28/2016 19:00:06 : g_vixInterfaceLogger:libvix.cpp:1775 <DEBUG> : [VFM_ESINFO] VixDiskLib: VixDiskLib_GetMetadataKeys: Retrieve metadata keys.

   01/28/2016 19:00:06 : g_vixInterfaceLogger:libvix.cpp:1775 <DEBUG> : [VFM_ESINFO] VixDiskLib: VixDiskLib_ReadMetadata: Retrieve metadata value.

   01/28/2016 19:00:06 : g_vixInterfaceLogger:libvix.cpp:1775 <DEBUG> : [VFM_ESINFO] VixDiskLib: VixDiskLib_ReadMetadata: Insufficient buffer size (has 0, need 9): Error 24 at 4611.

I did see an article about resource contention (Can't fine the article now) so I started taking my resource limits down for vcetner.
The original setting we had for resource limits was datastore = 3, so that allowed about 100 VMs to backup at a time.
We have now limited the resources to vCenter = 30 and snapshot = 30.
Unfortunatly the effect for these resource limts has doubled our backups times.

We have even tripled the amount of memory and doubled the CPU in the vetner appliances to try to help this issue and pinned the VMware external platform services controller to the same ESX host as the vCenter Appliance server.

We have cases opened with Veritas and VMWare at this point and no luck so far.  Looking for any advice.  I hope to be able to "guess" when this will happen again so I can collect VXMS logs again, but like I said it is very random.

Unfortunatly I cannot post the old VXMS files as they were purged already and I can't download them from the Veritas support case to post here.

Thanks.



 

5 REPLIES 5

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

VMWare vCetner Appliance (VCSA) was upgrades to 6.0u1b (was 6.0)

If I look at NBU Compatibility Guides, it seems that support for VCenter updates is not the same as for the base version.

 Firstly this doc: 

Statement of Support for NetBackup 7.x in a Virtual Environment:    http://www.veritas.com/docs/000006177 

Extract:

Notes on support for vSphere 6.0
NetBackup 7.6.1.1 adds support for vSphere 6.0, using VDDK 5.5.4.
NetBackup 7.7 uses VDDK 6.0 to support additional vSphere 6.0 functionality.

(Also see rest of above section regarding CBT backups.)

Next are the OS SCLs:

NetBackup 7 - 7.6  OS SCL :    http://www.veritas.com/docs/000040842
NBU 7.7 OS SCL: http://www.veritas.com/docs/000025229

Both docs show support for vSphere 6.0 and 6.0 U1.

Would you know if 6.0u1b is the same as 6.0 U1?

ianhoskins
Level 4

Hi Marianne, I had talked to Veritas to make sure u1b is supported and they checked with backline engineering and said it was.  I am now hearing that it MAY NOT be supported and they are checking with engineering again.

 

ianhoskins
Level 4

Just in case someone else needs this info:
 

Our veritas technical rep got ahodl of the PM responsible for VMWare.  This is the response.
Just wanted to get us all on the same page.  I checked with our Engineering team on this issue and here is their response:

“The issue with 5.5 update 3b was the disabling of SSLV3. This was done for 6.0 GA and required us to pick up VDDK 5.5.4 in 7.6.1.1. So 6.0 and later updates is already covered. There were a couple of things we had to verify in 6.0 u1a and there were no issues in u 1b.”

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
So, what do they say? Where to from here? Do you need to log a call with VMware to find out what in this update broke VADP backups?

ianhoskins
Level 4

Veritas is asking for more logs now and VMWare has already been involved.  We hope the 2 will get along and work thorugh this!  I will update as we get further along.