Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed

[Andre7]

I all.

I would like to ask for any help for the error message I am experiencing:

I have NetBackup 7.6.0.1 on Windows Server 2008 R2 Enterprise x64 with HP MSL2024 G3 Tape library

7/30/2014 10:26:56 AM - Info bpbrm(pid=123304) sedcims is the host to backup data from     
7/30/2014 10:26:56 AM - Info bpbrm(pid=123304) reading file list for client        
7/30/2014 10:26:56 AM - granted resource sedcims.NBU_CLIENT.MAXJOBS.sedcims
7/30/2014 10:26:56 AM - granted resource sedcims.NBU_POLICY.MAXJOBS.db_backup_class
7/30/2014 10:26:56 AM - granted resource 000001
7/30/2014 10:26:56 AM - granted resource HP.ULTRIUM5-SCSI.000
7/30/2014 10:26:56 AM - granted resource standard_backup_su
7/30/2014 10:26:56 AM - estimated 0 Kbytes needed
7/30/2014 10:26:56 AM - Info nbjm(pid=6596) started backup (backupid=sedcims_1406708816) job for client sedcims, policy db_backup_class, schedule DB_Full on storage unit standard_backup_su
7/30/2014 10:26:56 AM - started process bpbrm (123304)
7/30/2014 10:26:56 AM - connecting
7/30/2014 10:26:56 AM - started
7/30/2014 10:26:57 AM - Info bpbrm(pid=123304) starting bpbkar32 on client         
7/30/2014 10:26:57 AM - connected; connect time: 0:00:01
7/30/2014 10:26:58 AM - Info bpbkar32(pid=51832) Backup started           
7/30/2014 10:26:58 AM - Info bpbkar32(pid=51832) change time comparison:<disabled>          
7/30/2014 10:26:58 AM - Info bpbkar32(pid=51832) archive bit processing:<enabled>          
7/30/2014 10:26:58 AM - Info bpbkar32(pid=51832) not using change journal data for <H:\Backup>: not enabled    
7/30/2014 10:26:58 AM - Info bptm(pid=117308) start            
7/30/2014 10:26:58 AM - Info bptm(pid=117308) using 65536 data buffer size        
7/30/2014 10:26:58 AM - Info bptm(pid=117308) setting receive network buffer to 263168 bytes      
7/30/2014 10:26:58 AM - Info bptm(pid=117308) using 30 data buffers         
7/30/2014 10:26:58 AM - Info bptm(pid=117308) start backup           
7/30/2014 10:26:58 AM - Info bptm(pid=117308) Waiting for mount of media id 000001 (copy 1) on server sedcims.
7/30/2014 10:26:58 AM - mounting 000001
7/30/2014 10:27:02 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:02 AM - Warning bptm(pid=117308) media id 000001 load operation reported an error     
7/30/2014 10:27:02 AM - current media 000001 complete, requesting next resource Any
7/30/2014 10:27:06 AM - Info bptm(pid=117308) Waiting for mount of media id 000001 (copy 1) on server sedcims.
7/30/2014 10:27:06 AM - granted resource 000001
7/30/2014 10:27:06 AM - granted resource HP.ULTRIUM5-SCSI.001
7/30/2014 10:27:06 AM - granted resource standard_backup_su
7/30/2014 10:27:06 AM - mounting 000001
7/30/2014 10:27:10 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:10 AM - Warning bptm(pid=117308) media id 000001 load operation reported an error     
7/30/2014 10:27:10 AM - current media 000001 complete, requesting next resource Any
7/30/2014 10:27:14 AM - Info bptm(pid=117308) Waiting for mount of media id 000002 (copy 1) on server sedcims.
7/30/2014 10:27:14 AM - granted resource 000002
7/30/2014 10:27:14 AM - granted resource HP.ULTRIUM5-SCSI.000
7/30/2014 10:27:14 AM - granted resource standard_backup_su
7/30/2014 10:27:14 AM - end writing
7/30/2014 10:27:14 AM - mounting 000002
7/30/2014 10:27:18 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:18 AM - Warning bptm(pid=117308) media id 000002 load operation reported an error     
7/30/2014 10:27:18 AM - current media 000002 complete, requesting next resource Any
7/30/2014 10:27:22 AM - Info bptm(pid=117308) Waiting for mount of media id 000002 (copy 1) on server sedcims.
7/30/2014 10:27:22 AM - granted resource 000002
7/30/2014 10:27:22 AM - granted resource HP.ULTRIUM5-SCSI.001
7/30/2014 10:27:22 AM - granted resource standard_backup_su
7/30/2014 10:27:22 AM - mounting 000002
7/30/2014 10:27:25 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:25 AM - Warning bptm(pid=117308) media id 000002 load operation reported an error     
7/30/2014 10:27:25 AM - current media 000002 complete, requesting next resource Any
7/30/2014 10:27:29 AM - Info bptm(pid=117308) Waiting for mount of media id 000003 (copy 1) on server sedcims.
7/30/2014 10:27:29 AM - granted resource 000003
7/30/2014 10:27:29 AM - granted resource HP.ULTRIUM5-SCSI.000
7/30/2014 10:27:29 AM - granted resource standard_backup_su
7/30/2014 10:27:29 AM - end writing
7/30/2014 10:27:29 AM - mounting 000003
7/30/2014 10:27:33 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:33 AM - Warning bptm(pid=117308) media id 000003 load operation reported an error     
7/30/2014 10:27:33 AM - current media 000003 complete, requesting next resource Any
7/30/2014 10:27:37 AM - Info bptm(pid=117308) Waiting for mount of media id 000003 (copy 1) on server sedcims.
7/30/2014 10:27:37 AM - granted resource 000003
7/30/2014 10:27:37 AM - granted resource HP.ULTRIUM5-SCSI.001
7/30/2014 10:27:37 AM - granted resource standard_backup_su
7/30/2014 10:27:37 AM - mounting 000003
7/30/2014 10:27:41 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:41 AM - Warning bptm(pid=117308) media id 000003 load operation reported an error     
7/30/2014 10:27:41 AM - current media 000003 complete, requesting next resource Any
7/30/2014 10:27:44 AM - Info bptm(pid=117308) Waiting for mount of media id 000004 (copy 1) on server sedcims.
7/30/2014 10:27:44 AM - granted resource 000004
7/30/2014 10:27:44 AM - granted resource HP.ULTRIUM5-SCSI.000
7/30/2014 10:27:44 AM - granted resource standard_backup_su
7/30/2014 10:27:44 AM - end writing
7/30/2014 10:27:44 AM - mounting 000004
7/30/2014 10:27:48 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:48 AM - Warning bptm(pid=117308) media id 000004 load operation reported an error     
7/30/2014 10:27:48 AM - current media 000004 complete, requesting next resource Any
7/30/2014 10:27:53 AM - Info bptm(pid=117308) Waiting for mount of media id 000004 (copy 1) on server sedcims.
7/30/2014 10:27:53 AM - granted resource 000004
7/30/2014 10:27:53 AM - granted resource HP.ULTRIUM5-SCSI.001
7/30/2014 10:27:53 AM - granted resource standard_backup_su
7/30/2014 10:27:53 AM - mounting 000004
7/30/2014 10:27:56 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:27:56 AM - Warning bptm(pid=117308) media id 000004 load operation reported an error     
7/30/2014 10:27:56 AM - current media 000004 complete, requesting next resource Any
7/30/2014 10:28:01 AM - Info bptm(pid=117308) Waiting for mount of media id 000005 (copy 1) on server sedcims.
7/30/2014 10:28:01 AM - granted resource 000005
7/30/2014 10:28:01 AM - granted resource HP.ULTRIUM5-SCSI.000
7/30/2014 10:28:01 AM - granted resource standard_backup_su
7/30/2014 10:28:01 AM - end writing
7/30/2014 10:28:01 AM - mounting 000005
7/30/2014 10:28:04 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:28:04 AM - Warning bptm(pid=117308) media id 000005 load operation reported an error     
7/30/2014 10:28:04 AM - current media 000005 complete, requesting next resource Any
7/30/2014 10:28:08 AM - Info bptm(pid=117308) Waiting for mount of media id 000005 (copy 1) on server sedcims.
7/30/2014 10:28:08 AM - granted resource 000005
7/30/2014 10:28:08 AM - granted resource HP.ULTRIUM5-SCSI.001
7/30/2014 10:28:08 AM - granted resource standard_backup_su
7/30/2014 10:28:08 AM - mounting 000005
7/30/2014 10:28:12 AM - Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed     
7/30/2014 10:28:12 AM - Warning bptm(pid=117308) media id 000005 load operation reported an error     
7/30/2014 10:28:12 AM - current media 000005 complete, requesting next resource Any
7/30/2014 10:28:16 AM - Info bptm(pid=117308) EXITING with status 96 <----------        
7/30/2014 10:28:16 AM - Info bpbkar32(pid=51832) bpbkar waited 0 times for empty buffer, delayed 0 times.   
7/30/2014 10:28:19 AM - Info bpbkar32(pid=51832) done. status: 96: unable to allocate new media for backup, storage unit has none available
7/30/2014 10:28:19 AM - end writing
unable to allocate new media for backup, storage unit has none available(96)
7/30/2014 10:28:20 AM - Info bpbrm(pid=76432) Starting delete snapshot processing         
7/30/2014 10:28:22 AM - Info bpfis(pid=117736) Backup started           
7/30/2014 10:28:22 AM - Critical bpbrm(pid=76432) from client sedcims: cannot open C:\Program Files\Veritas\NetBackup\online_util\fi_cntl\bpfis.fim.sedcims_1406708816.1.0      
7/30/2014 10:28:22 AM - Info bpfis(pid=117736) done. status: 4207          
7/30/2014 10:28:22 AM - end operation
7/30/2014 10:28:22 AM - Info bpfis(pid=117736) done. status: 4207: Could not fetch snapshot metadata or state files 

Thanks a lot, I hope some one could give me help.

[INT_RND]

This error "Error bptm(pid=117308) error requesting media, TpErrno = Robot operation failed " is a harware error. It is often caused by physical robot crashing, failure of the drive to load the tape or bad media.

In this case I can tell from your log that the robot went through each tape it had available and marked them "Frozen" because they failed to load. This lead to the "status: 96: unable to allocate new media for backup". Netbackup does this to protect you from bad media. I think that the media is not the problem (unless you have a robot full of bad tape). 

To find the root cause:

1. Check the robot console for flags and collect error logs. Use the web GUI. Just enter the IP address of the robot in to your web brower.

2. Clean the drive that is being problematic. If you get an error during the cleaning tape load then you have bigger problems at hand.

3. Escalate any hardware issues you find to HP support.

4. Unfreeze the frozen tapes and retry the failed jobs

Information about frozen media is found here:

http://www.symantec.com/business/support/index?page=content&id=TECH21473

The manual for your hardware can be found here:

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/manualsResults/?sp4ts.oid=393630...

[Yasuhisa_Ishika]

In most cast, "robot operation failed" is caused by hardware issue, but it is better to check Windows Application Logs and System Logs.

BTW, I suppose you does not put barcode labels on tapes. Please check if you keep your media inventory  in NetBackup synchronized with that of physical tapes. For example, when NetBackup request tape A, NetBackup resolves its residence(robot number and slot number) refering media DB. If media DB tells slot X for media A but no tape in slot X physically, tape libray ignores load request from NetBackup.

[Marianne]

I would first of all try robtest to test robot operation.

It looks like none of the tapes can be mounted, so each tape that failed to mount was probably Frozen resulting in no more tapes that can be used for backups - status 96.

Please use robtest to see if these tapes can be mounted manually:

Robtest commands that can be used to test the SCSI functionality of a robot 
http://www.symantec.com/docs/TECH83129

[Pangal]

Check whether tapes are stuck in drive . You can do this simple by comparing recorded tapes in device monitor and comparing the tapes in drives in library .