09-30-2016 12:39 AM - edited 09-30-2016 12:43 AM
All of a sudden i see that all the driver connected to the media server shows AVR,, i can see the drive physically in OS level, also in NBU.
Robtest is also working , i can control the robot. i have bonunced the NBU service on media server.
Media server is active for both tape and disk. i am suspecting if its library . before i request for library reboot , just wanted to know if i have to check for something else too.
Media server is an appliance.
Solved! Go to Solution.
09-30-2016 02:43 AM
09-30-2016 01:00 AM
Well - robtest itself can cause drives to go in AVR mode.
Check and confirm that nobody has left an instance of robtest open.
Please show us output of these commands on the robot control host:
scan -changer
vmoprcmd
Check for errors in /var/log/messages.
09-30-2016 01:08 AM
What are the robot console/web interface showning ?
Can you see the robot in the OS ?
Are the robot control daemon(s) running ? tld & ltdcd for tld robots
How is the robot shown in netbackup under devices ?, have had an occurence where it was enabled NO which meant the drives stood as AVR. In that case it turned out there was an issue with the robotics (gripper&arm) in the robot.
09-30-2016 01:13 AM
attached the results..
Also checked /var/log/messages. , i see below messages..
Sep 30 03:03:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:03:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:03:48 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:04:03 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:04:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:04:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:04:48 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:05:03 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:05:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:05:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:05:48 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:06:03 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:06:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:06:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:06:48 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:07:03 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:07:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:07:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:07:48 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:08:03 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:08:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:08:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:08:48 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:09:03 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:09:18 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:09:33 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
Sep 30 03:09:49 xrxsenbumed02 avrd[3065]: MTIOCGET failed on HP.ULTRIUM6-SCSI.001 (device 1, /dev/nst36) ioctl failed, Input/output error
09-30-2016 01:38 AM
Double check drive config as well on this media server to ensure NBU is configured correctly with robot posistion for the drives:
tpconfig -l
There should be more in messages file - not only errors related to one drive.
A least during NBU restart there should be messages related to tldcd.
Is tldcd running? And tldd for each robotic drive?
Can you manually stop ltid (using stopltid and check that all Media Manager processes are stopped) and restart with 'ltid -v' ?
Which Media Manager processes are started at this point?
Use 'vmps' command to list Media Manager processes.
PS:
Device config seems to be in a bit of a mess with DOWN drives, Missing Path, etc.
And WAY too many tape drives attached to a single media server....
09-30-2016 02:01 AM
I went reading the messages , do see tldcd were shutdown abnormally during services restart..
Sep 29 18:40:09 xrxsenbumed02 tldcd[30991]: Daemon has terminated abnormally
Sep 29 18:48:55 xrxsenbumed02 ltid[30852]: Sending shutdown to tldcd daemon...
Sep 29 18:49:05 xrxsenbumed02 tldcd[3078]: starting tldcd
Sep 29 18:49:05 xrxsenbumed02 tldcd[3081]: emmlib_uninitialize failed, status=13
Sep 29 18:49:05 xrxsenbumed02 tldcd[3081]: System error occurred
Sep 29 18:49:05 xrxsenbumed02 tldcd[3081]: Daemon has terminated abnormally
i have tried restart ltid manually too.. below are mm process active
MM Processes
------------
root 2711 30250 0 Sep28 pts/2 00:00:00 [tldcd] <defunct>
root 2717 30250 0 Sep28 pts/2 00:00:00 [tldcd] <defunct>
root 3016 1 0 Sep29 ? 00:00:01 /usr/openv/volmgr/bin/ltid
root 3060 3016 0 Sep29 ? 00:00:00 tldd -v
root 3065 3016 0 Sep29 ? 00:00:02 avrd -v
root 8060 3060 0 03:53 ? 00:00:00 tldd -v
root 8096 8045 0 03:53 pts/4 00:00:00 robtest
root 8111 8096 0 03:54 pts/4 00:00:00 /usr/openv/volmgr/bin/tldtest -rn 0 -r /dev/sg38
root 30250 1 0 Sep03 pts/2 00:06:01 tldcd -v
root 30859 1 0 Sep29 ? 00:00:01 vmd -v
Theres too many backup active on this media server via disk. so need to wait if i have to restart the NBU services again.
09-30-2016 02:43 AM
09-30-2016 03:19 AM
Thanks Marianne ,
we broghtdown the services again, killed all the process. and restarted services.. its up and fine now.
too many services were hung. Thanks again for the help.
09-30-2016 04:05 AM