Huge number of Duplication job is failing with Error code 83 (media open error)

[Shas]

Hi All ,

Here I am with new problem :)

Huge number of Duplication job is failing with Error code 83

Below is the Detailed Staus of the job.

03/04/2014 02:39:12 - requesting resource LCM_PD_###_123-456-789
03/04/2014 02:39:19 - begin Duplicate
03/04/2014 02:39:19 - granted resource LCM_PD_`_123-456-789
03/04/2014 02:39:19 - started process RUNCMD (13904)
03/04/2014 02:39:19 - ended process 0 (13904)
03/04/2014 02:39:20 - requesting resource PD_###_123-456-789
03/04/2014 02:39:20 - reserving resource @aaace
03/04/2014 02:39:30 - reserved resource @aaace
03/04/2014 02:39:30 - granted resource MediaID=@aaacg;DiskVolume=PureDiskVolume;DiskPool=###-cbs-spa01;Path=PureDiskVolume;StorageServer=###-cbs-spa01;MediaServer=123-456-789
03/04/2014 02:39:30 - granted resource PD_###_123-456-789
03/04/2014 02:39:31 - requesting resource @aaace
03/04/2014 02:39:32 - granted resource MediaID=@aaace;DiskVolume=PureDiskVolume;DiskPool=123-cbs-spa01;Path=PureDiskVolume;StorageServer=123-cbs-spa01;MediaServer=123-456-789
03/04/2014 02:39:34 - Info Duplicate(pid=13904) Initiating optimized duplication from @aaace to @aaacg      
03/04/2014 02:39:35 - Info bpdm(pid=18472) started            
03/04/2014 02:39:35 - started process bpdm (18472)
03/04/2014 02:39:35 - Info bpdm(pid=18472) requesting nbjm for media         
03/04/2014 02:39:52 - Critical bpdm(pid=18472) image open failed: error 2060018: file not found     
03/04/2014 02:39:52 - Info 123-456-789(pid=18472) StorageServer=PureDisk:###-cbs-spa01; Report=PDDO Stats for (###-cbs-spa01): scanned: 1 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
03/04/2014 02:39:53 - Error bpduplicate(pid=13904) host 123-456-789 backup id brwacpr001_1395871043 optimized duplication failed, media open error (83). 
03/04/2014 02:39:53 - Info 123-456-789(pid=18472) StorageServer=PureDisk:123-cbs-spa01; Report=PDDO Stats for (123-cbs-spa01): scanned: 1 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
03/04/2014 02:39:53 - Error bpduplicate(pid=13904) db_IMAGE() failed: images are in process (1519)      
03/04/2014 02:39:53 - Error bpduplicate(pid=13904) Duplicate of backupid brwacpr001_1395871043 failed, media open error (83).    
03/04/2014 02:39:53 - Error bpduplicate(pid=13904) Status = no images were successfully processed.      
03/04/2014 02:39:53 - end Duplicate; elapsed time: 00:00:34
media open error(83)

NOTE:     I have replace the name of DR server name as ### and live master name with 123-456-789 and 123
    

I am confused from where to start looking after this issue.

[Marianne]

Critical bpdm(pid=18472) image open failed: error 2060018: file not found   

You need to find out what happened to deduped images. Why would they be 'not found'?

Also try to run a verify on image/backup id brwacpr001_1395871043 and let us know what the result is.

[sri_vani]

03/04/2014 02:39:52 - Critical bpdm(pid=18472) image open failed: error 2060018: file not found  

Please verify below TNs for error 2060018 along with 83

http://www.symantec.com/business/support/index?page=content&id=TECH141312

http://www.symantec.com/business/support/index?page=content&id=TECH146409

[Shas]

Hi Marianne,

Here is the output.

F:\Program Files\Veritas\NetBackup\bin\admincmd>bpverify -backupid brwacpr001_1395871043
Verify started 03/04/2014 13:48:02
INF - Verifying policy PD_BRIT-VIRTUAL-M-ENV-SYSTEM-IP, schedule Weekly (brwacpm001_1395871043), path "@aaace", created 03/26/2014 21:57:23.
INF - Verify of policy PD_BRIT-VIRTUAL-M-ENV-SYSTEM-IP, schedule Weekly (brwacpm001_1395871043) failed, media open error.

INF - Status = no images were successfully processed.

[Marianne]

Have a look at the 2 TNs that sri vani posted above and see if it applies here.

You will have to logon to PD server and check paths as per the TNs.

[Shas]

Logged a case with symantec ....

[Marianne]

Have you checked the dedupe paths (as described in the TNs) before logging the case?

[Mark_Solutions]

Is it just duplications that are failing? Are backups to that spa working?

Is this you own PureDisk setup or 5020 appliances?

Has this all been working OK up to now? Is so have anything changed recently?