VM backups are failing with Error code 40

[Ankit_J]

Hello Team,

I have a VM which is failing with error code 40 and the other Vm of the same datastore are running fine. The only difference is the VM which is working fine does the backup using SAN and the failed one using NBD. When I am trying to run the backup of VM in question with SAN it fails with the below error. Please suggest me further troubleshooting.

11/14/2013 14:56:56 - Info bptm (pid=25969) using 1048576 data buffer size
11/14/2013 14:56:56 - Info bptm (pid=25969) setting receive network buffer to 262144 bytes
11/14/2013 14:56:56 - Info bptm (pid=25969) using 256 data buffers
11/14/2013 14:56:57 - Info bptm (pid=25969) start backup
11/14/2013 14:56:59 - Error bpbrm (pid=25941) from client XXXXXXX: ERR - Error opening the snapshot disks using given transport mode: Status 23
11/14/2013 14:57:00 - Info bptm (pid=25969) backup child process is pid 26297
11/14/2013 14:57:00 - Error bptm (pid=25969) media manager terminated by parent process
11/14/2013 14:57:00 - begin writing
11/14/2013 14:57:09 - Info XXXXXX03 (pid=25969) StorageServer=PureDisk:XXXXXX03; Report=PDDO Stats for (XXXXXXXX03): scanned: 3 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
11/14/2013 14:57:11 - Info bpbkar (pid=0) done. status: 6: the backup failed to back up the requested files
11/14/2013 14:57:11 - end writing; write time: 0:00:11
the backup failed to back up the requested files  (6)

Regards

Ankit J

Netbackup Admin

[RamNagalla]

VM which is working fine does the backup using SAN and the failed one using NBD

this explines it..

do you have 902 port communication with the backup host to ESXi host?

check telnet <ESXihost having the VM> 902  ----> form the backup host.

attach the bpvmutil log from the backup host to this post.. that wil give you the esxi host name that needs to have the port 902 communication 

what is the OS of backup host?

what is the Vcenter/ESXi hosts version?

Netbackup version on backup host?

[Marianne]

One more thing to check in addition to Nagalla's excellent post:

Have you confirmed zoning and lun assignment of datastore to backup host?

[Ankit_J]

Thanks to both of you for replying ...

# The backup of other VM is working fine on the same datastore (with NBD as well as with SAN)

# When we use NBD mode for the VM having issue it fails after writing almost 300 GB of data and running after 2:30 Hour. There is no issues with the port 902. (But fails immediately with SAN mode)

Backup Host is Appliance
--------------
NetBackup 7.5.0.6 (Appliance)
SUSE Linux Enterprise Server 10 (x86_64)
VERSION = 10
PATCHLEVEL = 4
system-product-name = S5520UR
model = 5220
6.6.0.329Operating System :

[RamNagalla]

what is the error its giving when trying with SAN mode?

try increasing timeout vaules for NBD transport

[Marianne]

Seems we have 2 separate issues here:

SAN Backup - Status 6 (immediately).

NBD backup - Status 40 (After 2.5 hours).

About SAN backup - have you checked an confirmed zoning and LUN assignment?

You forgot to tell us what is the Vcenter/ESXi hosts version?

Is the Appliance just a Media server or Master/Media server?
If Media server - is there a firewall between Media and Master?
 

[rk1074]

check this out:

https://www-secure.symantec.com/connect/forums/vmware-backup-failed-using-san-transport-mode

Quoting the the Solution from Mark...

"Does that VM Client have all of its disks on the SAN LUNS? If any one of them are local to the ESX Server it will not work.

The vCenter credentials are the other thing that causes this but if all others work then i guess that is not the case.

I would check exactly where all vmdk's reside for the VM in question first - take a good look at it in vCenter to see what you can spot"