vm backup getting code 6 (the backup failed to ba...

nbustarter380 · ‎06-16-2016

Hi All,

We have a VM client (actually a few) that are getting code 6 errors. I am working on the one at a time, I am positing one here hopefully if this one gets solved it will help wiith others.

Vmware

STATUS CODE: 6 " the backup failed to back up the requested files"

The only error I found was in the bfis.log however doesn't seem like connect is and issue of course I could be incorrect. (see below)

vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6

01:09:47.152 [5792.3128] <8> init_connect_recs: [vnet_connect.c:738] vnet_cached_getaddrinfo() failed STAT=6 RV=11001 HOST=mbsec

01:09:47.152 [5792.3128] <8> connect_to_service: [vnet_connect.c:562] init_connect_recs() failed 6 0x6

01:09:47.152 [5792.3128] <8> vnet_connect_to_service: [vnet_connect.c:233] connect_to_service() failed 6 0x6

01:09:47.152 [5792.3128] <2> bprd_connect: vnet_connect_to_service(mbsec) failed: 6

01:09:47.152 [5792.3128] <2> bprd_read_text_file: bprd_connect(mbsec) failed, cannot connect on socket (25)

01:09:47.230 [5792.3128] <8> bpfis: WRN - VfMS error 10; see following messages:

fyi - In viewing todays details (right clicking on job status) I noticed the following

Error bpbrm (pid =82166) Error opening the snapshot disks using the given transport mode : nbd Status 23

I have attached the bpfis, bpcd and the bpbkar logs.

Windows 2012 R2 server

We put the Netbackup client 7.6.0.3 on client

Netbackup Master Linux Netbackup version 7.7.1

Thanks in advance for any help

nbustarter380 · ‎06-27-2016

Hello,

This is still an issue, has anyone had the code 6 issue with a VM and tried rebooting the vm to fix the issue?

Thanks in advance for any help!

Michael_G_Ander · ‎06-28-2016

Have you checked the technotes about VMWARE backup statuses 6 & 23 on the veritas support site ?

Is the snapshots created in the vmware ? Can usually be seen in the vsphere client

I would check the permission in vmware and the prerequistes in the vmware admin guide besides the technotes

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

nbustarter380 · ‎06-28-2016

Thanks Michael,

Yes, I checked the technotes for the code 6 and code 23 status. There are several clients getting to code 6 and most of them are getting snapshots created int vmware. The permissions in vmware seem fine.

I can tell you in the details I do see this error Error on one or two of them

bpbrm (pid=76169) from client XX-XXXX1X - Err -Error opening the snapshot disks using given transport mode:nbd status.

Error btm (pid=76203) media manager terminated by parent process

I am a vmbackup novice so I am still learning about possible casues for errors such as 6 and 156

Best Regards

Michael_G_Ander · ‎06-28-2016

How about the prerequistes ? no reason to troubleshoot something that cannot made to work

How about the vm snapshots on the failing vms are they created at all ?

If they are, you probably have an access problem of some kind, maybe nbd to a certain host/datastore from the backup host

if not, you will have troubleshoot the snapshot(s) inside the vms, application, file system

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

nbustarter380 · ‎06-29-2016

Hi Micheal,

Not sure about what you mean prerequistes? Do you mean for the VM backup?

On 7 out of 10 of the VM's failing with code 6 the vm snapshots are successful/created.

on 2 of them the vm's are not getting created.

In most of the VM's latest errors I see >>>Error opening the snapshot disks using given transport mode: Status 23

I think I need to consentrait on that error trying to research something you mentioned "nbd to a certain host/datastore from the backup host" Can I see that in Vcenter? we do have access.

Best Regards,

NBU35 · ‎06-30-2016

FIrst isolate the machines on which snapshots are generatting & of which snapshots are not preparing.

on machines whose spanshsots are getting generated and error is "Error opening the snapshot disks using given transport mode: Status 23" , check if you are able to telnet port 443, 902 of VCenter/ESXI hosts hosting these machines, from your master,media, backup host. you should be able to telnet these ports.

for remaining machines, check bpfis logs and try to generate snapshot from Vspehere client and check if it works or not.

nbustarter380 · ‎07-07-2016

Hello NBU35,

ThanksI was going to post this earlier however Veritas site maintenance prevented , what directory on the master I am using the telnet from (see below) I am getting Name or Service not known with the exception to one.

[root@vmbackup server admincmd]# telnet ty-arta21-v2-c

telnet: ty-arta21-v2-c: Name or service not known

ty-arta21-v2-c: Unknown host

telnet: ty-arta21-v2-c: Name or service not known

ty-arta21-v2-c: Unknown host

[root@vmbackup server ~]# cd /usr/openv/netbackup/bin/admincmd

[root@vmbackup server admincmd]# telnet ty-arta21-v2-c

telnet: ty-arta21-v2-c: Name or service not known

ty-arta21-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet ty-arta21-v2-c 443

telnet: ty-arta21-v2-c: Name or service not known

ty-arta21-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet ty-arta21-v2-c 902

telnet: ty-arta21-v2-c: Name or service not known

ty-arta21-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet oi-dneta24-v2-c

telnet: oi-dneta24-v2-c: Name or service not known

oi-dneta24-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet oi-dneta24-v2-c 902

telnet: oi-dneta24-v2-c: Name or service not known

oi-dneta24-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet oi-dneta24-v2-c 443

telnet: oi-dneta24-v2-c: Name or service not known

oi-dneta24-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet oi-dneta23-v2-c 443

telnet: oi-dneta23-v2-c: Name or service not known

oi-dneta23-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet oi-dneta23-v2-c 902

telnet: oi-dneta23-v2-c: Name or service not known

oi-dneta23-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet oi-dnetac1-v2-c 902

Trying 982.31.55.206...

telnet: connect to address 982.31.55.206: Connection refused

[root@vmbackup server admincmd]# telnet oi-dnetac1-v2-c 443

Trying 982.31.55.206...

Connected to oi-dnetac1-v2-c.

Escape character is '^]'.

Connection closed by foreign host.

[root@vmbackup server admincmd]# telnet fh-cneed2--v2-c 443

telnet: fh-cneed2--v2-c: Name or service not known

fh-cneed2--v2-c: Unknown host

[root@vmbackup server admincmd]# telnet fh-cneed2--v2-c 902

telnet: fh-cneed2--v2-c: Name or service not known

fh-cneed2--v2-c: Unknown host

[root@vmbackup server admincmd]# telnet fh-cneed2-v2-c 902

telnet: fh-cneed2-v2-c: Name or service not known

fh-cneed2-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet fh-cneed2-v2-c 443

telnet: fh-cneed2-v2-c: Name or service not known

fh-cneed2-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet ag-coars22-v2-c 443

telnet: ag-coars22-v2-c: Name or service not known

ag-coars22-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet ag-coars22-v2-c 902

telnet: ag-coars22-v2-c: Name or service not known

ag-coars22-v2-c: Unknown host

[root@vmbackup server admincmd]# telnet ag-coars21-v2-c 902

telnet: ag-coars21-v2-c: Name or service not known

ag-coars21-v2-c: Unknown host

[

root@vmbackup server admincmd]# telnet ag-coars21-v2-c 443

telnet: ag-coars21-v2-c: Name or service not known

ag-coars21-v2-c: Unknown host

[root@vmbackup server admincmd]#

Thanks

NBU35 · ‎07-07-2016

Are following your vCenters ?

ty-arta21-v2-c
ty-arta21-v2-c
oi-dneta24-v2-c
fh-cneed2--v2-c

if yes, then try to ping them from master, media, backup host.. it looks like their entry is missing from DNS.. looks like name resolution issue.

sclind · ‎07-07-2016

When we get this its almost always because the LUNs from VMware were not mapped to the Netbackup server doing the backup.

nbustarter380 · ‎07-11-2016

Thanks NBU35 and sclind for your responses

NBU35,

Sorry, Below are actually a few of the VM's that are getting the code 6 errors. I didn't know I should be trying to telnet the Vcenters from the master backup server.

ty-arta21-v2-c
ty-arta21-v2-c
oi-dneta24-v2-c
fh-cneed2--v2-c

Also just a to note there are other VM's that are backing up successfully using the same Vcenter does that make a difference?

Hi Sclind,

When we get this its almost always because the LUNs from VMware were not mapped to the Netbackup server doing the backup.

Okay, this would be something I would check with the VMware Admins on correct? If so It would be the VM's in question?

Thanks,

sclind · ‎07-11-2016

Yes - I check with the Vmware admin and the SAN admin.

nbustarter380 · ‎07-11-2016

Hi Sclind, Thanks I will update later.

Best Regards

nbustarter380 · ‎07-11-2016

Hi Sclind,

For the below, were there certain vm client backup/snapshots this was affecting?

"When we get this its almost always because the LUNs from VMware were not mapped to the Netbackup server doing the backup."

Or were all the vm clients getting code 6 with >>Error bpbrm(pid=XXXXXX) from client hostname: ERR - Error opening the snapshot disks using given transport mode:NBD Status 23

I am just checking because if the LUNs from VMware were not mapped to the Netbackup server doing the backup wouldn't that affect all the vm clients at least the ones using the same vcenter?

Sorry if I am not correct I am still a novice with VM backups.

Thanks

sclind · ‎07-11-2016

I am not an expert either but what I would say is this:

Whatever LUNs are presented to the ESX server need to also be presented to the media server performing the the Netbackup backups. For us that it a Windows 2008 server. When new LUNs are added to ESX I generally just reboot the Windows media server to pick up the new LUNs that have been presented to it. If the SAN guys forgets to map them then we get backups that fail rc=6.

Michael_G_Ander · ‎07-12-2016

Sorry, just reread the original post and discovered my just written comment made no sense.

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

Michael_G_Ander · ‎07-12-2016

If you are using NDB, the disks does not need to be mapped to the backup host as the backup is done through the esx host(s)

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

NBU35 · ‎07-13-2016

Luns should be maspped with read/write access on your backup host and rest depends which method in policy you choose for data transfer.

if your backups host is physical machine, get the luns zoned on physical machines and use SAN mode of transfer.

if your backup host is VM, it should reside in same DC and you can choose either hotadd or nbd mode. if you are using NBD mode of transfer then you need to get ports 443 & 902 open on your Vcenter/ESX hosts.

nbustarter380 · ‎07-18-2016

Hi All,

Thanks, for your reponses,

Hi NUB35, Just to be clear to check to see if the ports 443 and 902 are open I use the telnet command (example below) on the Master, Media and backup host correct?

Here is what I got when I used the telenet on the master for the esxhosts

[root@nbuserver5 ~]# telnet vubdellvhost-47.xxxx.xxx.xx.x

Trying 11.226.3.100...

telnet: connect to address 11.226.3.100: Connection timed out

[root@nbuserver5 ~]# telnet vubdellvhost-49.xxxx.xxx.xx.x

Trying 11.226.3.113...

telnet: connect to address 11.226.3.113: Connection timed out

[[root@nbuserver5 ~]# telnet vubdellvhost-46.xxxx.xxx.xx.x

Trying 11.226.3.95...

telnet: connect to address 11.226.3.95: Connection timed out

[root@nbuserver5 ~]# telnet vubdellvhost-50.xxxx.xxx.xx.x

Trying 11.226.3.112...

telnet: connect to address 11.226.3.112: Connection timed out

[root@nbuserver5 ~]# telnet vubdellvhost-47.xxxx.xxx.xx.x

Trying 11.226.3.100...

telnet: connect to address 11.226.3.100: Connection timed out

[[root@nbuserver5 ~]# telnet vubdellvhost-44.xxxx.xxx.xx.x

Trying 11.226.3.115...

telnet: connect to address 11.226.3.115: Connection timed out

I issued those from root was I suppose to issue the telnet command from a special directory? If so let me know.

Also should I have entered that command on the failing VM's that are getting the code 6 Error (bpbrm(pid=XXXXXX) from client hostname: ERR - Error opening the snapshot disks using given transport mode:NBD Status 23)

Thanks again

nbustarter380 · ‎07-25-2016

Hi,

Sorry for the delay, (veritas community migration) I did a telnet (from the master) on all the esx hosts of the vm’s that are getting the code 6 Error (bpbrm(pid=XXXXXX) from client hostname: ERR - Error opening the snapshot disks using given transport mode:NBD Status 23)

This was done from the Master server on ports 902 and 443

[root@nbuserver5 ~]# telnet vubdellvhost-41xxxx.xxx.xxx 902

Trying 11.226.3.91...

Connected to vubdellvhost-41xxxx.xxx.xxx.

Escape character is '^]'.

220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported, NFCSSL supported/t