VMWare backup failed

[T_Imegaline]

Hi all,

I'm using Netbackup 7.6.1.1 on Windows Server 2008 R2. I'm doing backup for virtual machines using vCenter 5.5u2.

Before the issue, Netbackup was 7.5 and VMWare was 5.0u1, since the upgrades i'm facing many backup problems, some backups fails for different reasons, and other hangs (which result in failure of the next planned backup once the schedule window finishes)

We are facing the issue since 1 month and the pressure from the customer is becoming ... you know

[Marianne]

I don't see any 'Known Issues' with NBU 7.6.1 and VMware in LBN:  http://www.symantec.com/docs/TECH227580 

All I can suggest is to troubleshoot errors one-by-one. 
And maybe log simultaneous calls with Symantec/Veritas and Vmware.

[T_Imegaline]

Hi,

Thank you for your reply.

I actually logged a case with Symantec, but unfortunately i don't renewed the support with VMWare.

Reegarding the error code 23, i found this on troubleshoot document:

• If you are using HotAdd, please make sure that your backup host is Virtual Machine and following conditions are satisfied:
  • The VM should not contain IDE disks.
  • Ensure that there are sufficient SCSI controllers attached on the Backup Host VM.
  • The Backup Host VM has access to datastores where VM being backed up resides.
  • The Backup Host VM and VM being backed up should be under the same datacenter.
  • If the previous backup failed, it might have left some disks of the backup VM attached to Backup Host. These disks need to be manually removed before attempting the next backup.

I checked and everything seems fine :(

[Marianne]

I personally think that hotadd is 'messy' and only suitable for a small environment.

Any chance you can use a physical client or media server as backup host?

If you are using VMware Intelligent Policies, configure Resource Limits to ensure ESX, vCenter, datastores, etc. are not overloaded.

[T_Imegaline]

Hi again,

First, i'm using a small environment about 50 VM.

The Backup server is a physical dedicated server, and regarding the ressource limits, it's already configured.

On vCenter side, no consistent logs, only failure notifications.

[Marianne]

If using physical server as VMware backup host, why mention hotadd? Which transport method is used in this environment? Can you confirm that you are using VMware Intelligent Policies? Resource Limits are only applied to VIPs. What limits are in place?

[Michael_G_Ander]

You mention status 23, is that your main error ?

Which transport mode are you using ?

What are you backing up to ? MSDP, Tape, something else ?

How was backup of VMWare 5.0u1 with NetBackup 7.6.1.1 ? Ask as this will give an indication of which upgrade is likely to have caused the issues

[ShelleyK]

Did a quick Google and found these links which may provide insight:

• VMware policy is failing with status 6 and 23: https://support.symantec.com/en_US/article.TECH211280.html
• VMware Backup Failure with NetBackup 7.5, Transport Mode Error 23, Status 6: https://www-secure.symantec.com/connect/forums/vmware-backup-failure-netbackup-75-transport-mode-error-23-status-6

Other ideas:

• You should enable logging for VMware backups, here's a good tech article: How to troubleshoot NetBackup for VMWare vSphere backup/restore in NetBackup 7.x - https://support.symantec.com/en_US/article.TECH77790.html
• Check your vCenter logs for the VMs which fail and when - do they fail at snapshot creation?
• Create new policy and backup only one VM and focus on that
• Remove snapshots for VMs which fail (if you can)

I use VMware Intelligent Policies and so far, they are awesome! If you're not using them, you may want to look into them further. We just had a huge VMware/vCenter upgrade and migration and NBU handled it great.

Also, you should take into account - and I believe this is an NBU "best practice" based on VMware input - is that you don't want any one ESX host or datastore "stressed" by snapshots/backup jobs. I have my Resource Limit settings to ESXserver=3 and Datastore=4. My policies are aligned with ESXservers.(One policy per ESXserver. YMMV.)

Good luck!
Michelle