cancel
Showing results for 
Search instead for 
Did you mean: 

NDMP Restore failing

smwoodcrafts
Level 6

Having an issue with an NDMP restore. It's failing with a 2813 error. Here is what I get in the detailed status:

2/10/2017 4:39:17 PM - Critical bptm(pid=6212) sts_read_image failed: error 2060001 bytes Read = 18339597817245633366one or more invalid arguments
2/10/2017 4:39:17 PM - Error ndmpagent(pid=4960) connection 01231908 ndmp_message_process_one failed, status = 18 (NDMP_XDR_DECODE_ERR)
2/10/2017 4:39:17 PM - Error ndmpagent(pid=4960) NDMP restore failed from path /ifs/LonProdMedia/lonvidediting/PROJECTS
2/10/2017 4:39:22 PM - Error ndmpagent(pid=4960) LONVIDNAS01.ap.org: Failed to read data from daemon: No such file or directoryError reading file data (31009/-1): No such file or directoryWrite error: Invalid argument: ifs/LonProdMedia/lonvidediting/PROJECTS/ZARA WORK IN PROGRESS/zara.prproj
2/10/2017 4:39:22 PM - Error ndmpagent(pid=4960) LONVIDNAS01.ap.org: Could not skip data giving up
2/10/2017 4:39:22 PM - Error ndmpagent(pid=4960) LONVIDNAS01.ap.org: Failed to read archive during DAR for index 0
2/10/2017 4:39:22 PM - Warning ndmpagent(pid=4960) LONVIDNAS01.ap.org: Error extracting entry {ZARA WORK IN PROGRESS/zara.prproj, /ifs/LonProdMedia/lonvidediting/PROJECTS/ZARA WORK IN PROGRESS/zara.prproj} at index 0 of NLIST
2/10/2017 4:39:22 PM - Warning ndmpagent(pid=4960) LONVIDNAS01.ap.org: Operation halted by DMA
2/10/2017 4:39:22 PM - Error ndmpagent(pid=4960) send error status = 12 (NDMP_EOF_ERR)
2/10/2017 4:39:23 PM - Info ndmpagent(pid=4960) done
2/10/2017 4:39:23 PM - Info ndmpagent(pid=4960) done. status: 27: child process killed by signal
2/10/2017 4:39:23 PM - Error bpbrm(pid=4128) client restore EXIT STATUS 27: child process killed by signal
2/10/2017 4:39:23 PM - restored image lonvidnas01.ap.org_1486279880 - (child process killed by signal(27)); restore time 00:00:14
2/10/2017 4:39:23 PM - end Restore; elapsed time: 00:00:15
NDMP policy restore error(2813)

I've looked into the solutions for the status code but can't see an issue. I would appreciate any help you could give.

16 REPLIES 16

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
The actual error is from ndmpagent, so, no use trying to look for answers in NBU documentation.

I may be wrong, but my interpretation is that the destination path is invalid.

You need to look in the documentation for your make/model NAS appliance. The various types of filers all have different rules for alternate path restore.

Thanks Marianne,

This is seems weird. I have been backing up this NDMP client for a while with no issues but the restore fails. I'm restoring to the original location. That what I don't get.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
There's a leading '/' missing here:
ifs/LonProdMedia.......

Maybe post screenshots of the last screen?

I assume you mean this line:

2/10/2017 4:39:22 PM - Error ndmpagent(pid=4960) LONVIDNAS01.ap.org: Failed to read data from daemon: No such file or directoryError reading file data (31009/-1): No such file or directoryWrite error: Invalid argument: ifs/LonProdMedia/lonvidediting/PROJECTS/ZARA WORK IN PROGRESS/zara.prproj

I see that there is no leading /, but where is this truing to read from? 

Not sure what you meant by last screen, though. Here is the screen from the restore status.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
I meant the last screen where you select/specify the restore destination.

Ok Sorry, feeling a little dense. Here is the screen shot.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Okay. Nothing strange there.

Next thing to look at is logs.
bprd on master server
ndmpagent, bpbrm and bptm on media server.
bprd and ndmpagent are the first ones I would look at.
bprd to make 100% sure the destination is the ndmp file and not unintendedly another host.
ndmpagent to see what the 'invalid parameters' are.

I tried it again and watched on the Isilon I'm restoring to. It creates the folder and then starts to write a file but before the file is completed it failes. I looked at the bprd and all I see is an empty text file. The others I pulled out the data pertaining to the issue and have uploaded them. This is not sinking in. Not sure where this error lies. Is this a permmissions issue? I back up this NDMP client with no trouble. 

Seems it does not like the embedded spaces in "ZARA WORK IN PROGRESS"

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
NetBackup must be restarted on the master to enable bprd log.
I will look at uploaded logs on my laptop a bit later. I can only view .txt files on my mobile phone.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Seems you are trying to restore to the master/media server and not the filer.

This is what I see in bpbrm log:

logparams: -restore -mt 0 -mst 6 -k 0 -c LONVIDNAS01.ap.org -hostname LONVIDNAS01.ap.org -rclnt lonbackup00 -rclnthostname lonbackup00 -browse_clnt LONVIDNAS01.ap.org

Did you choose lonbackup00 as destination client? 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Okay... now I am confused.

I'm reading your last post again:

"It creates the folder and then starts to write a file..."

So, maybe I am misinterpreting what I see in bpbrm.
I need bprd to know for sure.
Buth I also see only connection requests from IP 10.10.4.134 to same IP address: 10.10.4.134. 

Why would it create a folder if you are restoring back to the same location?
Was your last attempt something different to what you showed in the screenshot on Friday?
Or are you tying to restore to a different filer? 

What confuses me even more are the 'NAS' hostnames I see in bptm and bpbrm:

-c LONVIDNAS01.ap.org -cl LONVIDNAS01EDT

Two different filers?

PS:
ndmpagent is a unified log. You will need vxlogview command to get in plain English.

So sorry for the confusion. Let me explain. The requester deleted the folder and wanted it restored. I started the restore like any other restore source ans destination was the NDMP client. That failed so I tried again this time only chooing a couple of files in a sub folder of the main folder. That's when I saw the folder creation and data transferuntil it failed again. I tried a few more things.I tried to restore to a different folder on the same destination. My boss kept suggesting I restore it to the master server. I told him it wouldn't work but he was insistant so I tried it to show him. I must have pulled the ndmpagent events from that try. 

I attempted a restore of a file to start fresh and have taken the logs for that. I am attaching them. I did do vxlogview and had it grab the logs pertaining to the job id, but it came back with 244 corrupted log records so I opened the ndmpagent log in wordpad and searched the job id (restore id) and cut and pasted from there to the end.

BTW, in searching online, I saw a couple of the errors might be caused by DAR so I disabled it and started a restore of one file. Is this going to cause an issue?

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I honestly think you will be best off logging a Support call.

With our time differences, it means that I am only viewing VOX posts on my phone.

In light of you being around for so long, I was hoping that you would know how to use vxlogview or at least be prepared to look it up. 
We need vxlogview output for ndmpagent specifically. OID 134.

So, something like this: 
vxlogview -o 134 >C:\temp\ndmp.txt
(You can look at Commands manual for options to extract info for a specific period.)

bprd seems all good with browse_client, destination_client and destination_client_hostname all the same:

browse_client = LONVIDNAS01.ap.org
requesting_client = lonbackup00
destination_client = LONVIDNAS01.ap.org
requesting_client_hostname = lonbackup00
destination_client_hostname = LONVIDNAS01.ap.org


We now need vxlogview output for ndmpagent log. 

IMHO, DAR needs to be enabled for file-level restore.
There is obviously no harm in disabling it before you retry.
Check ndmpagent log with and without DAR. 

 

You are correct. I should know how to use this and I will do just that. To update you, disabling DAR allowed me to successfully restore the folder. It took a while, but it worked. I will work to be able to check the logs correctly.

Thanks for your help.