Instant Recovery "Failed to export a backup image on the media server" - socket open failed

[zfardea]

Trying to run instant recovery after a successful backup. 

I'm getting a socket error, but how can I tell what's wrong exactly? The media server doesn't have a firewall.

14:04:25.674 [8170] <2> process_publish_request: Found a backup image VR2-006.iLab.local_1422530711 supported for Instant Recovery for VMware
14:04:25.919 [8170] <8> retry_getaddrinfo_for_real: [vnet_addrinfo.c:1054] getaddrinfo() failed RV=-2 NAME=UNKNOWN SVC=0 errno=2
14:04:25.919 [8170] <8> retry_getaddrinfo: [vnet_addrinfo.c:894] retry_getaddrinfo_for_real failed RV=-2 NAME=UNKNOWN SVC=0
14:04:25.919 [8170] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1643] retry_getaddrinfo() failed RV=-2 NAME=UNKNOWN SVC=NULL
14:04:25.920 [8170] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.920 [8170] <8> vnet_is_local_host: [vnet_addrinfo.c:304] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME=UNKNOWN
14:04:25.921 [13119] <2> bprd: socket fd from accept() is 9
14:04:25.922 [13119] <2> listen_loop: request complete
14:04:25.923 [8181] <2> logconnections: 
14:04:25.923 [8181] <2> process_request: setsockopt SO_LINGER on 9 succeeded.
14:04:25.926 [8181] <2> connected_peer: Connection from host nbu76-vr2-01.ilab.local, 30.30.160.60, on non-reserved port 56241
14:04:25.926 [8181] <2> process_request: command C_READ_HOST_CONFIG (73) received
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1602] in failed file cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.927 [8181] <8> vnet_same_host_and_update: [vnet_addrinfo.c:2861] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME1=UNKNOWN
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1548] in failed cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.927 [8181] <8> gethostent_force: [vnet_addrinfo.c:2582] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME=UNKNOWN ADJUSTEDNAME=UNKNOWN
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1548] in failed cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.927 [8181] <8> vnet_same_host_and_update: [vnet_addrinfo.c:2861] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME1=UNKNOWN
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1548] in failed cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.927 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.927 [8181] <8> vnet_same_host_and_update: [vnet_addrinfo.c:2861] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME1=UNKNOWN
14:04:25.930 [8181] <8> vnet_legacy_cached_gethostbyname: [vnet_hosts.c:385] vnet_ipv4_gethostbyname failed, hostname=UNKNOWN, errno=1
14:04:25.930 [8181] <2> vnet_legacy_cached_gethostbyname: vnet_hosts.c.399: 140403385505568: Function failed: 6 0x00000006
14:04:25.931 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1548] in failed cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.931 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.931 [8181] <8> vnet_same_host_and_update: [vnet_addrinfo.c:2868] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME2=UNKNOWN
14:04:25.931 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1548] in failed cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.931 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.931 [8181] <8> vnet_same_host_and_update: [vnet_addrinfo.c:2861] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 NAME1=UNKNOWN
14:04:25.931 [8181] <8> vnet_cached_getaddrinfo_and_update: [vnet_addrinfo.c:1548] in failed cache ERR=-2 NAME=UNKNOWN SVC=NULL
14:04:25.931 [8181] <8> vnet_cached_getaddrinfo: [vnet_addrinfo.c:1271] vnet_cached_getaddrinfo_and_update() failed 6 0x6
14:04:25.931 [8181] <8> init_connect_recs: [vnet_connect.c:738] vnet_cached_getaddrinfo() failed STAT=6 RV=-2 HOST=UNKNOWN
14:04:25.931 [8181] <8> connect_to_service: [vnet_connect.c:562] init_connect_recs() failed 6 0x6
14:04:25.931 [8181] <8> vnet_connect_to_bpcd: [vnet_connect.c:297] connect_to_service() failed 6 0x6
14:04:25.931 [8181] <2> local_bpcr_connect: Can't connect to client UNKNOWN
14:04:25.931 [8181] <2> ConnectToBPCD: bpcd_connect_and_verify(UNKNOWN, UNKNOWN) failed: 48
14:04:25.932 [8170] <16> client_is_SAN_client: ReadHostConfig failure: client hostname could not be found (48)
14:04:26.117 [8170] <8> do_pbx_service: [vnet_connect.c:2152] via PBX 
14:04:26.117 [8170] <8> vnet_vnetd_connect_forward_socket_begin: [vnet_vnetd.c:443] VN_REQUEST_CONNECT_FORWARD_SOCKET 10 0xa
14:04:26.324 [8170] <8> vnet_vnetd_connect_forward_socket_begin: [vnet_vnetd.c:460] ipc_string 51987
14:04:29.386 [8170] <16> read_nbfsd_credentials: mangleConnect stat <25> mangleConnect: error:00000005:lib(0):func(0):DH lib
14:04:29.386 [8170] <16> bpcr_start_nbfsd_rqst: could not read nbfsd credentials
14:04:29.386 [8170] <16> NBFSDStorageProvider::exportBackupImage: bpcr_start_nbfsd_with_option_rqst failed with error code: 21
14:04:29.390 [8170] <16> process_publish_request: Failed to export a backup image on the media server nbu76-vr2-02.ilab.local
14:04:29.390 [8170] <16> process_request: process_publish_request failed - EC status = socket open failed

The restore command is this:

nbrestorevm -vmw -ir_activate -C VR2-006.ilab.local -R /root/rename_file_01 -temp_location scalevol01 -s 01/01/1980 -e 12/12/2020

And the rename file contents are this:

change vmname to restore01-vr2-06.ilab.local

[Marianne]

Which OS is running on media server?

Have you configured NFS and checked all other requirements?

Requirements as per NetBackup for VMware Administrator's Guide  :

• The virtual machine to restore must have been backed up from a VMware policy.
• The backup image must reside on NetBackup disk (basic, advanced or MSDP).
• The target ESX server for the restore must be at vSphere 5.0 or later.
• The restore host (NetBackup client) must be on Windows. (If backup host is NBU Appliance, the restore host can be a Windows VM)
• Restore host must have SERVER entry on NBU master
• The NFS Client service must be enabled on the ESX/ESXi host.
• The Services for Network File System (NFS) must be installed on the Windows restore host.
• The media server must use IPv4 (IPv6 is not supported).
• NetBackup requires login credentials for the vCenter server and the restore host.
• TCP ports 7394 and 111 between the target ESX server and the media server where image resides.

 Some more reading matter:

NetBackup 7.6 feature Briefing - Instant recovery for VMware 
http://www.symantec.com/docs/TECH211113

[Michael_G_Ander]

Looks to me like some kind of name resolution problem

Had a similar problem, in that case the -vmproxy and -vmserver options was the solution.

[zfardea]

I think all the preconditions are met, although some of these bullets are not quite clear:

• The backup image must reside on NetBackup disk (basic, advanced or MSDP) - is a basic storage (as a file system directory) in the media server good enough? I think so, but not sure.
• The restore host (NetBackup client) must be on Windows - not sure what restore host means in this context. I assume it's the media server where the backup is stored? I think this because I got a specific error about this initially when I tried to use the Linux master with IR.

Maybe it's IPv6, I will try to disable IPv6 on the media server and see if that helps. Anyway, for now I'm using a single master+media server on Windows as a simpler alternative.

[Marianne]

Important to verify all requirements - 

Basic Disk (volume/filesystem on media server) is supported.
The concept is similar to Disk Storage used for Exchange GRT backup.

Restore host is described in the manual mentioned above. Same function as backup host, but here the Windows Client or Media server that will perform the restore.
If media server is Unix/Linux, you need to provide a Windows host to act as restore host. 
This restore host must be added as Server in NBU Host Properties.
As per my post above, restore host can be Windows VM.

In a test environment, a single Windows master/media server that will be backup and restore host is the easiest to configure.

The manual contains a workaround for IPV4 hosts entry.

Double-check NFS requirements on all components.