All compatible drive paths are down but media is available

[kingwang]

2013-10-31 1:50:23 - requesting resource LCM_parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:50:23 - granted resource  LCM_parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:50:24 - begin Duplicate
2013-10-31 1:50:24 - end Duplicate; elapsed time 0:00:00
2013-10-31 1:50:24 - started process RUNCMD (pid=16046)
2013-10-31 1:50:24 - requesting resource parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:50:24 - requesting resource @aaaac
2013-10-31 1:50:24 - reserving resource @aaaac
2013-10-31 1:50:24 - Error nbjm (pid=12606) NBU status: 830, EMM status: All compatible drive paths are down, but media is available
2013-10-31 1:50:24 - Error nbjm (pid=12606) NBU status: 830, EMM status: All compatible drive paths are down, but media is available
2013-10-31 1:50:24 - Error nbjm (pid=12606) NBU status: 830, EMM status: All compatible drive paths are down, but media is available
All compatible drive paths are down but media is available  (2009)
 

[kingwang]

2013-10-31 1:35:21 - requesting resource LCM_parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:35:21 - granted resource  LCM_parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:35:22 - begin Duplicate
2013-10-31 1:35:22 - started process RUNCMD (pid=13639)
2013-10-31 1:35:22 - requesting resource parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:35:22 - requesting resource @aaaac
2013-10-31 1:35:22 - reserving resource @aaaac
2013-10-31 1:35:22 - resource @aaaac reserved
2013-10-31 1:35:22 - granted resource  PAR009
2013-10-31 1:35:22 - granted resource  HP.ULTRIUM5-SCSI.000
2013-10-31 1:35:22 - granted resource  parcsrv0006-hcart2-robot-tld-0
2013-10-31 1:35:22 - granted resource  MediaID=@aaaac;DiskVolume=PureDiskVolume;DiskPool=dp_disk_parcsrv0006;Path=PureDiskVolume;StorageServer=parcsrv0006;MediaServer=parcsrv0006
2013-10-31 1:35:23 - Info bptm (pid=13647) start
2013-10-31 1:35:23 - started process bptm (pid=13647)
2013-10-31 1:35:23 - Info bptm (pid=13647) start backup
2013-10-31 1:35:23 - ended process 0 (pid=13639)
2013-10-31 1:35:24 - Info bpdm (pid=13656) started
2013-10-31 1:35:24 - started process bpdm (pid=13656)
2013-10-31 1:35:24 - Info bpdm (pid=13656) reading backup image
2013-10-31 1:35:24 - Info bpdm (pid=13656) using 30 data buffers
2013-10-31 1:35:24 - Info bpdm (pid=13656) requesting nbjm for media
2013-10-31 1:35:24 - Info bptm (pid=13647) Waiting for mount of media id PAR009 (copy 2) on server parcsrv0006.
2013-10-31 1:35:24 - started process bptm (pid=13647)
2013-10-31 1:35:24 - mounting PAR009
2013-10-31 1:35:24 - Info bptm (pid=13647) INF - Waiting for mount of media id PAR009 on server parcsrv0006 for writing.
2013-10-31 1:35:27 - begin reading
2013-10-31 1:37:02 - Error bptm (pid=13647) error requesting media, TpErrno = Robot operation failed
2013-10-31 1:37:02 - Warning bptm (pid=13647) media id PAR009 load operation reported an error
2013-10-31 1:37:02 - current media PAR009 complete, requesting next media Any
2013-10-31 1:37:06 - Error bptm (pid=13647) NBJM returned an extended error status: All compatible drive paths are down but media is available (2009)
2013-10-31 1:37:06 - Info bptm (pid=13647) EXITING with status 252 <----------
2013-10-31 1:37:06 - Error nbjm (pid=12606) NBU status: 830, EMM status: All compatible drive paths are down, but media is available
2013-10-31 1:37:08 - Error bpdm (pid=13656) media manager terminated by parent process
2013-10-31 1:37:13 - Error bpduplicate (pid=13639) host parcsrv0006 backup id JN-NC_1383134408 read failed, media manager killed by signal (82).
2013-10-31 1:37:13 - Error bpduplicate (pid=13639) host parcsrv0006 backupid JN-NC_1383134408 write failed, extended error status has been encountered, check logs (252).
2013-10-31 1:37:13 - Error bpduplicate (pid=13639) Duplicate of backupid JN-NC_1383134408 failed, extended error status has been encountered, check logs (252).
2013-10-31 1:37:13 - Error bpduplicate (pid=13639) Status = no images were successfully processed.
2013-10-31 1:37:13 - end Duplicate; elapsed time 0:01:51
2013-10-31 1:37:18 - Info parcsrv0006 (pid=13656) StorageServer=PureDisk:parcsrv0006; Report=PDDO Stats for (parcsrv0006): read: 1925 KB, CR received: 5874 KB, CR received over FC: 0 KB, dedup: 0.0%
no images were successfully processed  (191)
 

[Yasuhisa_Ishika]

Check following points on parcsrv0006.

• status of tape drives using "tpconfig -d". If drives are down, try to set them up by "vmoprcmd -upbyname <drive_name>"
• any relevant messages in syslog(/var/log/messages).

Also check if this tape library is fully operational by its own management console.

[Marianne]

Please add VERBOSE entry to /usr/openv/volmgr/vm.conf, then restart NBU on parcsrv0006.

Use the Device Monitor or vmoprcmd to UP the drives.

If drives go down again, reason will be logged to syslog. 
If you tell us which Unix we can tell you location of the file (e.g. /var/adm/messages on Solaris).
 

[Will_Restore]

>>2013-10-31 1:37:02 - Error bptm (pid=13647) error requesting media, TpErrno = Robot operation failed

Check for stuck robot arm or other tape library failure. 

[NBdmecha]

Could it be a SAN problem as anyone of your drives is ready?

By the way, that kind of errors (media complete):

2013-10-31 1:37:02 - current media PAR009 complete, requesting next media Any

use to happen when netbackup doesn´t realize that the robot has already mount the tape, and after the mount timeout error 52 is displayed. In your case the error appears 2 minutes after the mount command so unless your mount timeout is 2 minutes it can´t be the cause.

I would check SAN connectivity and tapes stuck in drives.

If everything is OK after checking, UP drives in Device Monitor and give it a try. If fails, let us know.

[zhangwei8607]

Solved my issue, actually before run tpconifg command, you need do the followings:

nbu5240.Support> Maintenance
maintenance-!> /opt/Symantec/sdcssagent/IPS/sisipsoverride.sh
To override the policy and disable protection, enter your login password.
Password:

Choose the type of override that you wish to perform:
1. Override Prevention except for Self-Protection
2. Override Prevention Completely
Choice? [1] 1

maintenance-!> elevate
nbu5240:/home/maintenance # tpconfig -d

nbu5240:/home/maintenance# vmoprcmd -upbyname theName