NDMP backup Issue with vnxe3150 nas box in Netbackup 7.5.0.6

[Priyeranjan]

Hi Team,

I am facing NDMP backup issue with vnxe3150 nas box and my master and media server is Aix 6.1 and Netbackup version is 7.5.0.6

For information : we have two volumes and one is completing and another one is failing in full backup. Even cumilative is completing successfully and only full backup is failign for same volume.

Pls help me.

Thank you.

Priyeranjan

08/13/2014 14:02:08 - Info nbjm (pid=15335630) starting backup job (jobid=893420) for client estfls02, policy emcnas_estfls02_1d_win_tape, schedule monthly_full_aix
08/13/2014 14:02:10 - estimated 0 kbytes needed
08/13/2014 14:02:10 - Info nbjm (pid=15335630) started backup (backupid=estfls02_1407927729) job for client estfls02, policy emcnas_estfls02_1d_win_tape, schedule monthly_full_aix on storage unit cbsest01-ndmp-acs
08/13/2014 14:02:11 - started process bpbrm (pid=8388854)
08/13/2014 14:02:13 - connecting
08/13/2014 14:02:14 - connected; connect time: 0:00:00
08/13/2014 14:02:16 - mounting ET0075
08/13/2014 14:02:30 - current media ET0075 complete, requesting next media Any
08/13/2014 14:02:31 - Info bpbrm (pid=8388854) estfls02 is the host to backup data from
08/13/2014 14:02:31 - Info bpbrm (pid=8388854) reading file list from client
08/13/2014 14:02:32 - Info bpbrm (pid=8388854) starting ndmpagent on client
08/13/2014 14:02:33 - Info ndmpagent (pid=10354874) Backup started
08/13/2014 14:02:33 - Info bpbrm (pid=8388854) bptm pid: 5046276
08/13/2014 14:02:33 - Info bptm (pid=5046276) start
08/13/2014 14:02:33 - Info bptm (pid=5046276) using 30 data buffers
08/13/2014 14:02:33 - Info bptm (pid=5046276) using 65536 data buffer size
08/13/2014 14:02:34 - Info bptm (pid=5046276) start backup
08/13/2014 14:02:35 - Info bptm (pid=5046276) Waiting for mount of media id ET0075 (copy 1) on server cbsest01.
08/13/2014 14:02:48 - Error bptm (pid=5046276) error requesting media, TpErrno = Robot operation failed
08/13/2014 14:02:48 - Warning bptm (pid=5046276) media id ET0075 load operation reported an error
08/13/2014 14:02:49 - granted resource  ET0147
08/13/2014 14:02:49 - granted resource  ES-LTO104-2771
08/13/2014 14:02:49 - granted resource  cbsest01-ndmp-acs
08/13/2014 14:02:50 - end writing
08/13/2014 14:02:50 - mounting ET0147
08/13/2014 14:03:08 - Info bptm (pid=5046276) Waiting for mount of media id ET0147 (copy 1) on server cbsest01.
08/13/2014 14:03:45 - mounted ET0147; mount time: 0:00:55
08/13/2014 14:04:03 - Info bptm (pid=5046276) media id ET0147 mounted on drive index 3, drivepath c160t0l0, drivename ES-LTO104-2771, copy 1
08/13/2014 14:04:20 - positioning ET0147 to file 1
08/13/2014 14:04:47 - positioned ET0147; position time: 0:00:27
08/13/2014 14:04:47 - begin writing
08/13/2014 14:05:21 - Info ndmpagent (pid=10354874) 0 entries sent to bpdbm
08/13/2014 14:08:21 - Info ndmpagent (pid=10354874) 5000 entries sent to bpdbm
08/13/2014 14:13:31 - Info ndmpagent (pid=10354874) 10000 entries sent to bpdbm
08/13/2014 14:15:36 - Info ndmpagent (pid=10354874) 15000 entries sent to bpdbm
08/13/2014 14:16:55 - Info ndmpagent (pid=10354874) 20000 entries sent to bpdbm
08/13/2014 14:18:16 - Info ndmpagent (pid=10354874) 25000 entries sent to bpdbm
08/13/2014 14:19:24 - Info ndmpagent (pid=10354874) 30000 entries sent to bpdbm
08/13/2014 14:19:42 - Info ndmpagent (pid=10354874) 35000 entries sent to bpdbm
08/13/2014 14:20:13 - Info ndmpagent (pid=10354874) 40000 entries sent to bpdbm
08/13/2014 14:22:51 - Info ndmpagent (pid=10354874) 45000 entries sent to bpdbm
08/13/2014 14:22:56 - Info ndmpagent (pid=10354874) 50000 entries sent to bpdbm
08/13/2014 14:26:30 - Info ndmpagent (pid=10354874) 55000 entries sent to bpdbm
08/13/2014 14:27:44 - Info ndmpagent (pid=10354874) 60000 entries sent to bpdbm
08/13/2014 14:31:29 - Info ndmpagent (pid=10354874) 65000 entries sent to bpdbm
08/13/2014 14:32:24 - Info ndmpagent (pid=10354874) 70000 entries sent to bpdbm
08/13/2014 14:32:46 - Info ndmpagent (pid=10354874) 75000 entries sent to bpdbm
08/13/2014 14:33:18 - Info ndmpagent (pid=10354874) 80000 entries sent to bpdbm
08/13/2014 14:33:52 - Info ndmpagent (pid=10354874) 85000 entries sent to bpdbm
08/13/2014 14:34:24 - Info ndmpagent (pid=10354874) 90000 entries sent to bpdbm
08/13/2014 14:34:48 - Info ndmpagent (pid=10354874) 95000 entries sent to bpdbm
08/13/2014 14:34:59 - Info ndmpagent (pid=10354874) 100000 entries sent to bpdbm
08/13/2014 14:35:21 - Info ndmpagent (pid=10354874) 105000 entries sent to bpdbm
08/13/2014 14:35:58 - Info ndmpagent (pid=10354874) 110000 entries sent to bpdbm
08/13/2014 14:36:25 - Info ndmpagent (pid=10354874) 115000 entries sent to bpdbm
08/13/2014 14:36:45 - Info ndmpagent (pid=10354874) 120000 entries sent to bpdbm
08/13/2014 14:37:18 - Info ndmpagent (pid=10354874) 125000 entries sent to bpdbm
08/13/2014 14:37:57 - Info ndmpagent (pid=10354874) 130000 entries sent to bpdbm
08/13/2014 14:52:47 - Info ndmpagent (pid=10354874) 135000 entries sent to bpdbm
08/13/2014 15:14:57 - Info ndmpagent (pid=10354874) 140000 entries sent to bpdbm
08/13/2014 15:16:24 - Info ndmpagent (pid=10354874) 145000 entries sent to bpdbm
08/13/2014 15:17:06 - Info ndmpagent (pid=10354874) 150000 entries sent to bpdbm
08/13/2014 15:17:39 - Info ndmpagent (pid=10354874) 155000 entries sent to bpdbm
08/13/2014 15:17:40 - Info ndmpagent (pid=10354874) 160000 entries sent to bpdbm
08/13/2014 15:18:31 - Info ndmpagent (pid=10354874) 165000 entries sent to bpdbm
08/13/2014 15:18:54 - Info ndmpagent (pid=10354874) 170000 entries sent to bpdbm
08/13/2014 15:19:01 - Info ndmpagent (pid=10354874) 175000 entries sent to bpdbm
08/13/2014 15:19:13 - Info ndmpagent (pid=10354874) 180000 entries sent to bpdbm
08/13/2014 15:19:21 - Info ndmpagent (pid=10354874) 185000 entries sent to bpdbm
08/13/2014 15:19:27 - Info ndmpagent (pid=10354874) 190000 entries sent to bpdbm
08/13/2014 15:19:37 - Info ndmpagent (pid=10354874) 195000 entries sent to bpdbm
08/13/2014 15:20:13 - Info ndmpagent (pid=10354874) 200000 entries sent to bpdbm
08/13/2014 15:20:36 - Info ndmpagent (pid=10354874) 205000 entries sent to bpdbm
08/13/2014 15:22:48 - Info ndmpagent (pid=10354874) 210000 entries sent to bpdbm
08/13/2014 15:24:12 - Info ndmpagent (pid=10354874) 215000 entries sent to bpdbm
08/13/2014 15:31:16 - Info ndmpagent (pid=10354874) 220000 entries sent to bpdbm
08/13/2014 15:36:07 - Info ndmpagent (pid=10354874) 225000 entries sent to bpdbm
08/13/2014 16:04:03 - Info ndmpagent (pid=10354874) 230000 entries sent to bpdbm
08/13/2014 16:04:13 - Info ndmpagent (pid=10354874) 235000 entries sent to bpdbm
08/13/2014 16:04:23 - Info ndmpagent (pid=10354874) 240000 entries sent to bpdbm
08/13/2014 16:05:00 - Info ndmpagent (pid=10354874) 245000 entries sent to bpdbm
08/13/2014 16:06:59 - Info ndmpagent (pid=10354874) 250000 entries sent to bpdbm
08/13/2014 16:12:00 - Info ndmpagent (pid=10354874) 255000 entries sent to bpdbm
08/13/2014 16:13:45 - Info ndmpagent (pid=10354874) 260000 entries sent to bpdbm
08/13/2014 16:30:00 - Info ndmpagent (pid=10354874) 265000 entries sent to bpdbm
08/13/2014 16:33:21 - Info ndmpagent (pid=10354874) 270000 entries sent to bpdbm
08/13/2014 16:51:32 - Info ndmpagent (pid=10354874) 275000 entries sent to bpdbm
08/13/2014 16:51:36 - Info ndmpagent (pid=10354874) 280000 entries sent to bpdbm
08/13/2014 16:51:53 - Info ndmpagent (pid=10354874) 285000 entries sent to bpdbm
08/13/2014 16:52:18 - Info ndmpagent (pid=10354874) 290000 entries sent to bpdbm
08/13/2014 16:57:21 - Info ndmpagent (pid=10354874) 295000 entries sent to bpdbm
08/13/2014 16:59:17 - Info ndmpagent (pid=10354874) 300000 entries sent to bpdbm
08/13/2014 17:00:18 - Info ndmpagent (pid=10354874) 305000 entries sent to bpdbm
08/13/2014 17:11:37 - Info ndmpagent (pid=10354874) 310000 entries sent to bpdbm
08/13/2014 17:15:33 - Info ndmpagent (pid=10354874) 315000 entries sent to bpdbm
08/13/2014 17:17:11 - Info ndmpagent (pid=10354874) 320000 entries sent to bpdbm
08/13/2014 17:18:31 - Info ndmpagent (pid=10354874) 325000 entries sent to bpdbm
08/13/2014 17:38:29 - Info ndmpagent (pid=10354874) 330000 entries sent to bpdbm
08/13/2014 17:38:30 - Info ndmpagent (pid=10354874) 335000 entries sent to bpdbm
08/13/2014 17:48:51 - Info ndmpagent (pid=10354874) 340000 entries sent to bpdbm
08/13/2014 19:47:42 - Info ndmpagent (pid=10354874) 355000 entries sent to bpdbm
08/13/2014 20:39:51 - current media ET0147 complete, requesting next media Any
08/13/2014 20:39:52 - current media -- complete, awaiting next media Any. Waiting for resources.
          Reason: Drives are in use, Media server: cbsest01,
          Robot Type(Number): TLD(1), Media ID: N/A, Drive Name: N/A,
          Volume Pool: Eastern_NDMP, Storage Unit: cbsest01-ndmp-acs, Drive Scan Host: N/A,
          Disk Pool: N/A, Disk Volume: N/A
08/13/2014 20:39:56 - Info ndmpagent (pid=10354874) eastndmp: End of tape reached. Load next tape.
08/13/2014 20:40:57 - granted resource  ET0079
08/13/2014 20:40:57 - granted resource  ES-LTO104-2771
08/13/2014 20:40:57 - granted resource  cbsest01-ndmp-acs
08/13/2014 20:40:58 - end writing; write time: 6:36:11
08/13/2014 20:40:58 - mounting ET0079
08/13/2014 20:41:16 - Info bptm (pid=5046276) Waiting for mount of media id ET0079 (copy 1) on server cbsest01.
08/13/2014 20:41:35 - mounted ET0079; mount time: 0:00:37
08/13/2014 20:41:49 - positioning ET0079 to file 1
08/13/2014 20:41:54 - Info bptm (pid=5046276) media id ET0079 mounted on drive index 3, drivepath c160t0l0, drivename ES-LTO104-2771, copy 1
08/13/2014 20:42:00 - positioned ET0079; position time: 0:00:11
08/13/2014 20:42:00 - begin writing
08/13/2014 20:46:24 - Info ndmpagent (pid=10354874) 360000 entries sent to bpdbm
08/13/2014 21:26:59 - Info ndmpagent (pid=10354874) 365000 entries sent to bpdbm
08/13/2014 21:28:57 - Info ndmpagent (pid=10354874) 370000 entries sent to bpdbm
08/13/2014 21:29:58 - Info ndmpagent (pid=10354874) 375000 entries sent to bpdbm
08/13/2014 21:55:05 - Info ndmpagent (pid=10354874) 380000 entries sent to bpdbm
08/13/2014 22:16:47 - Info ndmpagent (pid=10354874) 385000 entries sent to bpdbm
08/13/2014 22:16:48 - Info ndmpagent (pid=10354874) 390000 entries sent to bpdbm
08/13/2014 22:22:25 - Info ndmpagent (pid=10354874) 395000 entries sent to bpdbm
08/13/2014 22:39:06 - Info ndmpagent (pid=10354874) 400000 entries sent to bpdbm
08/13/2014 22:41:16 - Info ndmpagent (pid=10354874) 405000 entries sent to bpdbm
08/13/2014 22:41:18 - Info ndmpagent (pid=10354874) 410000 entries sent to bpdbm
08/13/2014 22:41:22 - Info ndmpagent (pid=10354874) 415000 entries sent to bpdbm
08/13/2014 22:41:29 - Info ndmpagent (pid=10354874) 420000 entries sent to bpdbm
08/13/2014 22:41:30 - Info ndmpagent (pid=10354874) 425000 entries sent to bpdbm
08/13/2014 22:41:34 - Info ndmpagent (pid=10354874) 430000 entries sent to bpdbm
08/13/2014 22:41:36 - Info ndmpagent (pid=10354874) 435000 entries sent to bpdbm
08/13/2014 22:41:43 - Info ndmpagent (pid=10354874) 440000 entries sent to bpdbm
08/13/2014 22:42:00 - Info ndmpagent (pid=10354874) 445000 entries sent to bpdbm
08/13/2014 22:47:04 - Info ndmpagent (pid=10354874) 450000 entries sent to bpdbm
08/13/2014 22:47:06 - Info ndmpagent (pid=10354874) 455000 entries sent to bpdbm
08/13/2014 22:47:08 - Info ndmpagent (pid=10354874) 460000 entries sent to bpdbm
08/13/2014 22:47:27 - Info ndmpagent (pid=10354874) 465000 entries sent to bpdbm
08/13/2014 22:47:29 - Info ndmpagent (pid=10354874) 470000 entries sent to bpdbm
08/13/2014 22:47:32 - Info ndmpagent (pid=10354874) 475000 entries sent to bpdbm
08/13/2014 22:47:34 - Info ndmpagent (pid=10354874) 480000 entries sent to bpdbm
08/13/2014 22:47:39 - Info ndmpagent (pid=10354874) 485000 entries sent to bpdbm
08/13/2014 22:48:07 - Info ndmpagent (pid=10354874) 490000 entries sent to bpdbm
08/13/2014 23:04:24 - Info ndmpagent (pid=10354874) 495000 entries sent to bpdbm
08/13/2014 23:06:26 - Info ndmpagent (pid=10354874) 500000 entries sent to bpdbm
08/13/2014 23:06:28 - Info ndmpagent (pid=10354874) 505000 entries sent to bpdbm
08/13/2014 23:06:45 - Info ndmpagent (pid=10354874) 510000 entries sent to bpdbm
08/13/2014 23:48:07 - current media ET0079 complete, requesting next media Any
08/13/2014 23:48:08 - current media -- complete, awaiting next media Any. Waiting for resources.
          Reason: Drives are in use, Media server: cbsest01,
          Robot Type(Number): TLD(1), Media ID: N/A, Drive Name: N/A,
          Volume Pool: Eastern_NDMP, Storage Unit: cbsest01-ndmp-acs, Drive Scan Host: N/A,
          Disk Pool: N/A, Disk Volume: N/A
08/13/2014 23:48:21 - Info ndmpagent (pid=10354874) eastndmp: End of tape reached. Load next tape.
08/13/2014 23:48:43 - granted resource  ET0009
08/13/2014 23:48:43 - granted resource  ES-LTO104-2771
08/13/2014 23:48:43 - granted resource  cbsest01-ndmp-acs
08/13/2014 23:48:43 - end writing; write time: 3:06:43
08/13/2014 23:48:43 - mounting ET0009
08/13/2014 23:49:02 - Info bptm (pid=5046276) Waiting for mount of media id ET0009 (copy 1) on server cbsest01.
08/13/2014 23:49:30 - mounted ET0009; mount time: 0:00:47
08/13/2014 23:49:49 - Info bptm (pid=5046276) media id ET0009 mounted on drive index 3, drivepath c160t0l0, drivename ES-LTO104-2771, copy 1
08/13/2014 23:49:54 - positioning ET0009 to file 1
08/13/2014 23:50:17 - positioned ET0009; position time: 0:00:23
08/13/2014 23:50:17 - begin writing
08/14/2014 01:33:58 - Info ndmpagent (pid=10354874) 520000 entries sent to bpdbm
08/14/2014 01:34:02 - Info ndmpagent (pid=10354874) 525000 entries sent to bpdbm
08/14/2014 01:34:10 - Info ndmpagent (pid=10354874) 530000 entries sent to bpdbm
08/14/2014 02:01:51 - Info ndmpagent (pid=10354874) 535000 entries sent to bpdbm
08/14/2014 02:02:03 - Info ndmpagent (pid=10354874) 540000 entries sent to bpdbm
08/14/2014 02:08:31 - Info ndmpagent (pid=10354874) 545000 entries sent to bpdbm
08/14/2014 02:10:09 - Error bpbrm (pid=8388854) db_FLISTsend failed: file read failed (13)
08/14/2014 02:10:10 - Info ndmpagent (pid=0) done
08/14/2014 02:10:10 - Error bptm (pid=5046276) media manager terminated by parent process
08/14/2014 02:10:11 - Error ndmpagent (pid=10354874) NDMP backup failed, path = /common
08/14/2014 02:10:12 - Error ndmpagent (pid=10354874) estfls02: Medium error
08/14/2014 02:10:12 - Warning ndmpagent (pid=10354874) estfls02: Write failed on archive volume 1
08/14/2014 02:10:12 - Info ndmpagent (pid=10354874) estfls02: server_archive: emctar vol 1, 274968 files, 0 bytes read, 266074980352 bytes written
08/14/2014 02:10:12 - Error ndmpagent (pid=10354874) estfls02: Backup is aborted.
08/14/2014 02:12:11 - end writing; write time: 2:21:54
08/14/2014 02:12:30 - Info ndmpagent (pid=0) done. status: 13: file read failed
08/14/2014 02:12:30 - Error ndmpagent (pid=10354874) connection 11021d3d0 ndmp_message_process_one failed, status = 18 (NDMP_XDR_DECODE_ERR)
08/14/2014 02:12:30 - Error ndmpagent (pid=10354874) connection 1102205b0 ndmp_message_process_one failed, status = 18 (NDMP_XDR_DECODE_ERR)
file read failed  (13)

[SymTerry]

Collect the Unified log for 134 (NDMPAgent) and also look at the bpbrm  and filer logs.

It could be timing out. TECH21246 has the steps to change the NDMP timeout.