NDMP Failure - file read failed(13)

Open an SSH session to the EMC NDMP device.
Run the command nas_storage -c -a.

If the result of the command returns a storage device error for example error 5017: storage health check failed, contact EMC support.

 

https://www.veritas.com/support/en_US/article.TECH209515

 

I would start with ndmpagent log to see ndmp errors. bptm (at logging level 3) will show until when data was received. bpbrm (at logging level 3) will show until when metadata was received and if backup was terminated because of timeout. Can you show us a couple of lines before the 3/3/2016 9:39:05 PM timestamp?

Hi Marianne,

I have enabled the log after witnessing the error in detailed logs.

3/4/2016 10:36:28 PM - Error bpbrm(pid=26864) db_FLISTsend failed: file read failed (13)       
3/4/2016 10:36:28 PM - Error ndmpagent(pid=28288) terminated by parent process         
3/4/2016 10:36:28 PM - Info ndmpagent(pid=0) done            
3/4/2016 10:36:28 PM - Error ndmpagent(pid=28288) NDMP backup failed, path = /root_vdm_1/HongKong1       
3/4/2016 10:36:58 PM - Error ndmpagent(pid=28288) SNG2NAS02: Medium error          
3/4/2016 10:36:59 PM - Warning ndmpagent(pid=28288) SNG2NAS02: Write failed on archive volume 1      
3/4/2016 10:36:59 PM - Info ndmpagent(pid=28288) SNG2NAS02: server_archive: emctar vol 1, 332560 files, 0 bytes read, 186872496128 bytes written
3/4/2016 10:36:59 PM - Error ndmpagent(pid=28288) SNG2NAS02: Backup is aborted.         
3/4/2016 10:37:06 PM - Info ndmpagent(pid=0) done.  status: 13: file read failed       
3/4/2016 10:37:06 PM - end writing; write time: 3:38:27
file read failed(13)

getting below logs for ndmpagent log. I am unable to find time stamp.

- 228654 0 228654 0 346406 0 0 6 33188 32770 32775 228654 1430302194 1242356934 1364804255 0000002c240ef25e000000000003289d 207005,32:ConnectionToBrm::SendFileHistory,5
0,51216,134,134,694063,1457110366129,63900,60800,0:,145:FH TO BRM: Ino - 220318 0 220318 0 346407 0 0 6 33188 32770 32775 220318 1430302194 1242356950 1364804255 0000002c241275ad000000000003289e 207006,32:ConnectionToBrm::SendFileHistory,5
0,51216,134,134,694064,1457110366129,63900,60800,0:,145:FH TO BRM: Ino - 253086 0 253086 0 346408 0 0 6 33188 32770 32775 253086 1430302194 1242356964 1364804255 0000002c2415d8fc000000000003289f 207007,32:ConnectionToBrm::SendFileHistory,5
0,51216,134,134,694065,1457110366129,63900,60800,0:,145:FH TO BRM: Ino - 265952 0 265952 0 346409 0 0 6 33188 32770 32775 265952 1430302194 1242356976 1364804255 0000002c2419bc4b00000000000328a0 207008,32:ConnectionToBrm::SendFileHistory,5

 

 

 

You need to use vxlogview to read ndmpagent. I am not close to a system to look up command usage. Please do so in Commands manual (or maybe Google). You can specify start and end times with vxlogview to narrow down the time span.

vxlogview -p nb -o 134 -d all -b "3/4/2016 10:30:00 PM"

This gives you the readable (date & time) of ndmpagent logs (OID=134) from that 3/4/2016 10:30PM onwards, you can give it an end range by -e , if you know when the job ends or fails.

Btw, your error looks more like a tape media issue:

3/4/2016 10:36:28 PM - Error ndmpagent(pid=28288) NDMP backup failed, path = /root_vdm_1/HongKong1       
3/4/2016 10:36:58 PM - Error ndmpagent(pid=28288) SNG2NAS02: Medium error  

I would use a different tape or tape drive to test the backup again.

Have you checked the EMC hardware logs per my post above?

 

mnolan for the win!