NDMP backups are failing with 2009

so are you taking the local NDMP backups.. where the Tape Drives are connected to filer?

if yes...

does filer able to see the tape Drives... if yes.. 

what is the output of below command 

tpautoconf -probe <filername>

tpautoconf -verify <filername>

does it displaying any Drives?

if filer is not able to see the tape Drives, you need to check with filer team to make sure filer is able to detect the tape drives before you start looking into netbackup level.

Yes, i able to see the drives by passing the command - probe and it is connecting to the ndmp host from master server under -verify command

it is a good sign... 

show us the detail status of the faild job.. and also below commands output 

tpautoconf -probe <filername>

tpconfig -d

vmoprcmd -d

attacach these outputs as file to this post.

Prebe_output is attached from master server

i cannot attach toconfig and vmoprcmd outpupt form master server so i will try attach in the next comments

tpconfig_output from master server,

i will upload vmoprcmd output in next comment

here we go, we got all the outputs as requested, kindly let me know what else i can do for investigation please

is the down drive... and it only one Drive configured for this filer..

0   COPDD2V1_PB_31       hcart3 TLD(14)  DRIVE=31
      /dev/rmt0.1                                                      UP
      nrst370h (datal5150)                                             DOWN

and it is also visible in probe command

  Device "nrst370h" attributes=(0x4) RAW
    DENSITY=LTO3 400GB
    ELECTRICAL_NAME=COPTAPA4801:338.56
    SERIAL_NUMBER=74900100F9
    WORLD_WIDE_NAME=WWN[2:000:002188:4c597c]
    ALIAS 0=st370

so no issues in connectivity end..

please proceed as below

vmoprcmd -reset 0

vmoprcmd -up 0

wait for 2-5 min make sure drive is up, if it is up, try the backups again and test it

if backups are running fine..you are good..

if the Drive does not come up, or went down again. remove the drive and reconfigure it and then test the backups.

Thanks a lot, I reset the drive and now it is in up state but it is not mounting the tape. would you suggest to removing and readding the drive will be the best at this point?

as it is not mounting the tape, and it was down previously i would suggest to check the hardware function..

try mounting  and dismounting the tape into Drive by using the robtest, and make sure it is successfull.. 

if you find any issues with Mounting, check if it has any stuck tape.. and raise a hardware call.

if you are able to do the mounding and dismounting without any issues...  go ahead and reconfigure it and see how it goes.. 

Thanks, Drive went down again after sometime. May i know how to remove and readd this particular drive in master server please?