the restore failed to recover the requested files

[e_win]

We restored an NDMP backup from tape and after waiting for 15 hours to read and verify files, it returns error status code 5:

Detailed error in activity monitor is

04/17/2013 11:30:13 - begin Restore
04/17/2013 11:31:55 - media needed: P30932
04/17/2013 11:31:55 - media needed: P30927
04/17/2013 11:31:55 - restoring from image atp4nfs02P_1365609608
04/17/2013 11:31:56 - connecting
04/17/2013 11:31:57 - connected; connect time: 0:00:00
04/17/2013 11:31:57 - started process bptm (pid=10658)
04/17/2013 11:32:00 - started process bptm (pid=10658)
04/17/2013 11:32:00 - mounting P30927
04/17/2013 11:31:58 - requesting resource P30927
04/17/2013 11:31:59 - granted resource  P30927
04/17/2013 11:31:59 - granted resource  atp4nfs02p-dr
04/17/2013 11:32:48 - mounted P30927; mount time: 0:00:48
04/17/2013 11:32:48 - positioning P30927 to file 3
04/17/2013 11:33:19 - positioned P30927; position time: 0:00:31
04/17/2013 11:33:20 - begin reading
04/17/2013 23:51:18 - current media P30927 complete, requesting next media atp4nfs02p-dr:P30932
04/17/2013 23:51:19 - awaiting resource atp4nfs02p-dr:P30932.
          A pending request has been generated for this resource request.
          Operator action may be required. Pending Action: All drives down,
          Media ID: P30932, Barcode: P30932, Density: hcart3, Access Mode: Read,
          Action Drive Name: N/A, Action Media Server: N/A, Robot Type(Number): 8(0),
          Volume Group: 000_00000_TLD, Action Acs: N/A, Action Lsm: N/A
04/17/2013 23:51:20 - awaiting resource atp4nfs02p-dr:P30932. Waiting for resources.
          Reason: Drives are in use, Media server: atp3backupsp01,
          Robot Type(Number): TLD(0), Media ID: N/A, Drive Name: N/A,
          Volume Pool: N/A, Storage Unit: N/A, Drive Scan Host: N/A,
          Disk Pool: N/A, Disk Volume: N/A
04/17/2013 23:52:20 - started process bptm (pid=10658)
04/17/2013 23:52:20 - mounting P30932
04/17/2013 23:52:19 - granted resource  P30932
04/17/2013 23:52:19 - granted resource  atp4nfs02p-dr
04/17/2013 23:53:33 - mounted P30932; mount time: 0:01:13
04/17/2013 23:53:33 - positioning P30932 to file 1
04/17/2013 23:53:33 - positioned P30932; position time: 0:00:00
04/17/2013 23:53:33 - begin reading
04/18/2013 03:23:00 - Error ndmpagent (pid=10657) NDMP restore failed from path /vol/test
04/18/2013 03:23:16 - end reading; read time: 3:29:43
04/18/2013 03:24:13 - Error bpbrm (pid=10647) client restore EXIT STATUS 5: the restore failed to recover the requested files
04/18/2013 03:24:13 - restored from image atp4nfs02P_1365609608; restore time: 15:52:18
04/18/2013 03:24:13 - end Restore; elapsed time 15:54:00
the restore failed to recover the requested files (5)

Need Help...plss,..

Tnx.

[Marianne]

Any reason why you are not mentioning NBU version?

Please post restore log - it should be on the server where you started the restore under usr/openv/netbackup/logs/user_ops/<user-name>/logs.

bptm log on the media server may also be helpful.
please rename log to bptm.txt and post as File attachment.
 

[e_win]

Hi Marianne,

 Sorry, I forgot, its 6.5

/usr/openv/netbackup/logs/user_ops/root/logs says

root@atp3backupsp01 # more jbp-10256366169412767861000000009-Cvaacu
40960 0 0 0 40960 0 0 0 0 31 /vol/test/ublox/prod/fusion-mx/ 1365609608 1365609608
 

root@atp3backupsp01 # more jbp-10256366169412767861000000009-Cvaacu.chg
change /vol/test/ublox/prod/ to /vol/arch/restore/ublox

root@atp3backupsp01 # more jbp-10256366169412767861000000009-Cvaacu.log
Restore started 04/17/2013 11:30:12

11:31:55 (91555.xxx) Media id P30932 is needed for the restore.
11:31:55 (91555.xxx) Media id P30927 is needed for the restore.

11:31:55 (91555.001) Restoring from image created Thu Apr 11 00:00:08 2013
11:31:58 (91555.001) INF - If Media id P30927 is not in a robotic library administrative interaction may be required to satisfy this mount request.
11:32:00 (91555.001) INF - Waiting for mount of media id P30927 on server atp3backupsp01 for reading.
11:32:48 (91555.001) INF - Waiting for positioning of media id P30927 on server atp3backupsp01 for reading.
11:33:20 (91555.001) INF - Beginning restore from server atp3backupsp01 to client atp4nfs02P.
11:33:20 (91555.001) INF - Restoring NDMP files from /vol/test/ublox/prod/ to /vol/arch/restore/ublox
11:33:22 (91555.001) DAR enabled
11:33:22 (91555.001) Restoring ublox/prod/fusion-mx/ to /vol/arch/restore/ublox/fusion-mx/
23:51:18 (91555.001) INF - If Media id P30932 is not in a robotic library administrative interaction may be required to satisfy this mount request.
23:52:20 (91555.001) INF - Waiting for mount of media id P30932 on server atp3backupsp01 for reading.
23:53:33 (91555.001) INF - Waiting for positioning of media id P30932 on server atp3backupsp01 for reading.
23:53:33 (91555.001) INF - Continuing restore from server atp3backupsp01 to client atp4nfs02P.
03:23:00 (91555.001) NDMP restore failed from path /vol/test
03:24:13 (91555.001) Status of restore from image created Thu Apr 11 00:00:08 2013 = the restore failed to recover the requested files

03:24:14 (91555.xxx) INF - Status = the restore failed to recover the requested files.
 

i also attached bptm...

Regards,

e_win

[sksujeet]

This is the error you are getting in the bptm log:

ndmpagent[0] reports failure - this path

Checked google and found this TN though there are some others but this matches it most. Please check

http://www.symantec.com/docs/TECH191311
 

Also are you redirecting the restore of NDMP to the same box or different box. If you are restoring NDMP restore to different box with different make and model it will not work.

[e_win]

Hi sazz,

Yes we are restoring NDMP to another NAS box...but both are shared to the client (the source and destination)...

source - tesnetapps01:/vol/test

destination - nas03-r200:/vol/arch

If you are restoring NDMP restore to different box with different make and model it will not work. -- why?

do you know some document that can help?

Regards,

e_win

[sksujeet]

sure check the documents below

The server to which you want to redirect the restored data must be from the same vendor as the server from which the data was backed up. 

http://www.symantec.com/docs/HOWTO23318

http://www.symantec.com/docs/TECH63918

[e_win]

Hi sazz,

Thank you very much but Both filers are netapp and we've also tried before, restoring NDMP from FaS3020 to R200 filers.

Regards,

e_win

[Marianne]

NDMP protocol is different between NAS vendors. That is the main reason why restore to different vendor NAS does not work.

[e_win]

Both are netapp and we are able to restore before from this scenario...

I checked bprd and found this entries:

11:27:04.203 [10265] <2> get_type_of_client_list_restore: db_getCLIENT() failed: no entity was found (227)
11:27:04.204 [10265] <2> get_type_of_client_list_restore: db_getCLIENT_by_hostname() failed: no entity was found (227)
11:27:04.204 [10265] <2> get_type_of_client_free_browse: db_getCLIENT() failed: no entity was found (227)
11:27:04.204 [10265] <2> get_type_of_client_free_browse: db_getCLIENT_by_hostname() failed: no entity was found (227)
 

[sksujeet]

hmm.. restoring to same manufacturer's device should work but if both devices from the same manufacture are in different data format it will not. Could you please check the vendor if they are compatible with each other as far as data format and OS is concerned.

Also please try to restore a small file to same box on both the boxes. Then try to restore small file from NAS A to NAS B at different paths and then NAS B to NAS A at different paths to check if we are hitting the compatible matrix. Also make sure you are selecting the path correctly and giving correct information while restoring the data.

Please run a restore again and post the detailed status, bptm log and ndmp log.

[e_win]

both are unix file system format.

setup is like this, the data required is from the two tapes (P30932 and P30927) but we do not restore it to the same volume (/test/ublox/prod/fusion-mx) which is in FAS3020, rather we restore it to (/testarchive/restore/ublox/fusion-mx) which is in R200.

We just want to determine why it returned exit status code:5

As for my readings, Error code:5 is too general and I cannot link all the logs that points out why it failed, is it network?, corrupt tape?, drive down? wrong entry of source or destination? hostname and IP resolution?

Actually, we are trying to restore it now to the same volume but different foder (/vol/test/ublox/prod/fusion-mx_restore/fusion-mx/)...and since its NDMP it takes a longer time. Regarding the last backup, 15 hours later before we know that it is failed. Now I'm checking what cause to failed but to no avail.

[Marianne]

NBU 6.5.6 has 'cleaned up' status code 5 and is now giving more meaningful errors and status codes.

With older versions of NBU, you need to go into all sorts of logs to find the real reason.

PLEASE upgrade your environment ASAP - NBU 6.x reached EOSL in Oct 2012.