Forum Discussion

john10's avatar
john10
Level 6
12 years ago
Solved

Robotic library is down on server, Media Server

 

Hello All,

Currently we are using NBU 7.0.1 with 2008 as Master/Media server, and backup are not running fine in job details it was showing as "Robotic library is down on server, Media Server" and when i executed "vmoprcmd -d" the drive status are showing as in "AVR" mode , i recycled NBU services couple of time even though same AVR and i tried to load the media by clicking on media but is has give an error like "error connecting to Oprd on "X". network protocol error(39)" , and after some time im able to load the media's, and some outputs are below, hope any one can help.

Note:-Rebooted the Library twice.

 

Stuck job screen shot:-

Stuck Job details:-

1/10/2014 1:31:23 PM - requesting resource ko20w1000-hcart2-robot-tld-0
1/10/2014 1:31:23 PM - requesting resource ko20w1000.NBU_CLIENT.MAXJOBS.ko20w1000
1/10/2014 1:31:23 PM - requesting resource ko20w1000.NBU_POLICY.MAXJOBS.FULL_KO20NT100
1/10/2014 1:31:23 PM - awaiting resource ko20w1000-hcart2-robot-tld-0 Reason: Tape media server is not active, Media Server: ko20w1000,
     Robot Number: 0, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,
     Volume Pool: DataStore, Storage Unit: ko20w1000-hcart2-robot-tld-0, Drive Scan Host: N/A
    
1/10/2014 1:37:07 PM - awaiting resource ko20w1000-hcart2-robot-tld-0 Reason: Robotic library is down on server, Media Server: ko20w1000,
     Robot Number: 0, Robot Type: TLD, Media ID: N/A, Drive Name: N/A,

 

Vmoprcmd -d Output:-


C:\Program Files\Veritas\Volmgr\bin>vmoprcmd.exe -d

                                PENDING REQUESTS

                                     <NONE>

                                  DRIVE STATUS

Drv Type   Control  User      Label  RecMID  ExtMID  Ready   Wr.Enbl.  ReqId
  0 hcart2   AVR                -                     No       -         0
  1 hcart2   AVR                -                     No       -         0

                             ADDITIONAL DRIVE STATUS

Drv DriveName            Shared    Assigned        Comment
  0 IBM.ULT3580-HH5.000   No       -
  1 IBM.ULT3580-HH5.001   No       -
 

  • its looking like an libraary issue..

    if you have webconsole access to the library please check in the there if it is showing any alerts...

    if not try to check the robot functionlaty from the webconsole by mounthing the tapes into drive.

    you may need to check with library vendor.

    reboot of library also worth trying...

  • I am wondering about your screenshot showing 'I/O device error' when you ran scan command. 

    Have you done anything to fix this? Like rebooting the robot and the server?
    Is scan output now showing correct robot and drive output?

    Did you complete the device wizard config? Including restart of NBU Device Manager?

    Please show us config and status after device wizard has completed (including restart):

    tpconfig -l 

    vmoprcmd -d

  • Agree with Marianne above. Scan is nothing to do with NetBackup, so if you get I/O errors there is a problem at hardware/connectivity/OS level.

  • The device may very well be there, but it is not possible to perform any operation.

    Check the robot, I bet some red LED is flashing in the service required panel.

    • Check robot path for obstructions
    • Check the robot is fully initialized (door opened ?)

     

  • Hello All,

    We had a call with hardware vendor, and got replace the SAS cable and now the issue was fixed, thanks for your support and suggestions.

20 Replies

Replies have been turned off for this discussion
  • The device may very well be there, but it is not possible to perform any operation.

    Check the robot, I bet some red LED is flashing in the service required panel.

    • Check robot path for obstructions
    • Check the robot is fully initialized (door opened ?)

     

  • If scan does not show the robot (and /or) drives, you have a problem outside of NetBackup.

    scan simply sends some scsi commands to the library/ drives to ask them to provide information about themselves.  Apart from a bit of formatting (to make the putput look nice) what you 'should' see in the scan command output is SENT from the hardware device.

    Therefore, if it doesn't work, with there is a connectivity issue with the path to the device, or the device has a fault.

    I appreciate scan is provided by Symantec, but it is not a NetBackup command - it simply sends scsi commands to the devices that are configured in the Operating System.

  • Hello All,

    Thanks for your suggestions.

    and Marianne below are the out puts, and we rebooted the Library couple of time and, as the server is prodution we are going to Reboot it today and i will update the status once it was done.

     

    Tpconfig _L output:-

     

     

    tpconfig -l.png

    and when itried to configure the tape drives and Robot from Wizard below is the error i faced.

     

    config error.png

  • tpconfig -l shows that a robot is configured with no tape drives. 

    Please show us the screen at the back?

    We can see something saying '.... or drives removed from robot'.

    Please use Device Monitor to rescan OS for devices, then run 'scan' again and show us the output. 
    (Please copy the text in the output  - easier to post than a screenshot...)

    Have you checked OS Event Viewer logs for device errors? 
    Check System and Application logs.

    PS:
    Is this server running other applications as well?
    Your robot control host should ideally not run any applications other than NBU. 
    The reason for this recommendation is because of what you are experiencing right now:
    Device errors need to be checked and fixed at OS level. Windows servers often need a reboot to fix such issues. If NBU is the only application on the server, a reboot should not be a problem.

  • Hi All,

    Sorry for the delay, i was on Leave.

    The Masrer/Media server and Tape Library was reboted still the issue was not fixed, when i excute the Robtest and scan its showing error, and when i try to delete the Storage devices and try to configure from Wizard its taking very Long time to scan the devices.

    Below are the error messages.

     

    inv_0.png


    C:\Program Files\Veritas\Volmgr\bin>vmoprcmd.exe -d

                                    PENDING REQUESTS

                                         <NONE>

                                      DRIVE STATUS

    Drv Type   Control  User      Label  RecMID  ExtMID  Ready   Wr.Enbl.  ReqId
      0 hcart2   AVR                -                     No       -         0
      1 hcart2   AVR                -                     No       -         0

                                 ADDITIONAL DRIVE STATUS

    Drv DriveName            Shared    Assigned        Comment
      0 IBM.ULT3580-HH5.000   No       -
      1 IBM.ULT3580-HH5.001   No       -

    C:\Program Files\Veritas\Volmgr\bin>tpconfig.exe -l
    Device Robot Drive       Robot                    Drive                Device
    Type     Num Index  Type DrNum Status  Comment    Name                 Path
    robot      0    -    TLD    -       -  -          -                    {8,0,3,1}
    drive      -    0 hcart2    -      UP  -          IBM.ULT3580-HH5.000  {8,0,3,0}
    drive      -    1 hcart2    -      UP  -          IBM.ULT3580-HH5.001  {8,0,4,0}

    C:\Program Files\Veritas\Volmgr\bin>




    C:\Program Files\Veritas\Volmgr\bin>scan
    ************************************************************
    *********************** SDT_TAPE    ************************
    *********************** SDT_CHANGER ************************
    ************************************************************
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    DeviceIoControl() error on bus 0, target 3, lun 1 (1117:The request could not be performed because of an I/O device error. )
    ------------------------------------------------------------
    Device Name  : "Tape0"
    Passthru Name: "Tape0"
    Volume Header: ""
    Port: 8; Bus: 0; Target: 3; LUN: 0
    Inquiry    : "IBM     ULT3580-HH5     C7R3"
    Vendor ID  : "IBM     "
    Product ID : "ULT3580-HH5     "
    Product Rev: "C7R3"
    Serial Number: "1068082526"
    WWN          : ""
    WWN Id Type  : 0
    Device Identifier: "IBM     ULT3580-HH5     1068082526"
    Device Type    : SDT_TAPE
    NetBackup Drive Type: 10
    Removable      : Yes
    Device Supports: SCSI-6
    Flags : 0x0
    Reason: 0x0
    ------------------------------------------------------------
    Device Name  : ""
    Passthru Name: ""
    Volume Header: ""
    Port: 8; Bus: 0; Target: 3; LUN: 1
    Inquiry    : "IBM     3573-TL         B.20"
    Vendor ID  : "IBM     "
    Product ID : "3573-TL         "
    Product Rev: "B.20"
    Serial Number: "00X2U78Z0500_LL0"
    WWN          : ""
    WWN Id Type  : 0
    Device Identifier: "IBM     3573-TL         00X2U78Z0500_LL0"
    Device Type    : SDT_CHANGER
    NetBackup Robot Type: 8
    Removable      : Yes
    Device Supports: SCSI-5
    Number of Drives : 0
    Number of Slots  : 0
    Number of Media Access Ports: 0
    Flags : 0x0
    Reason: 0x0
    ------------------------------------------------------------
    Device Name  : "Tape1"
    Passthru Name: "Tape1"
    Volume Header: ""
    Port: 8; Bus: 0; Target: 4; LUN: 0
    Inquiry    : "IBM     ULT3580-HH5     C7R3"
    Vendor ID  : "IBM     "
    Product ID : "ULT3580-HH5     "
    Product Rev: "C7R3"
    Serial Number: "1068082530"
    WWN          : ""
    WWN Id Type  : 0
    Device Identifier: "IBM     ULT3580-HH5     1068082530"
    Device Type    : SDT_TAPE
    NetBackup Drive Type: 10
    Removable      : Yes
    Device Supports: SCSI-6
    Flags : 0x0
    Reason: 0x0

    C:\Program Files\Veritas\Volmgr\bin>
     

  • Repeat of my question asked on 13 January:

    Have you checked OS Event Viewer logs for device errors? 
    Check System and Application logs.

    Looking at scan output, the robot seems to be faulty. 
    The robot itself does not know about any tape drives or slots:

    Inquiry    : "IBM     3573-TL         B.20"

    Vendor ID  : "IBM     "
    Product ID : "3573-TL         "
    Product Rev: "B.20"
    Serial Number: "00X2U78Z0500_LL0"
    WWN          : ""
    WWN Id Type  : 0
    Device Identifier: "IBM     3573-TL         00X2U78Z0500_LL0"
    Device Type    : SDT_CHANGER
    NetBackup Robot Type: 8
    Removable      : Yes
    Device Supports: SCSI-5
    Number of Drives : 0
    Number of Slots  : 0

    Number of Media Access Ports: 0
    Flags : 0x0
    Reason: 0x0

    You need to log a call with the robot vendor.

  • Hi Marianne,

    Below are the error mesg found in Eventvwr

     

    TLD(0) mode_sense ioctl() failed: The operation completed successfully.

  • More evidence that you need to log a call with your hardware vendor...

  • Hello All,

    We had a call with hardware vendor, and got replace the SAS cable and now the issue was fixed, thanks for your support and suggestions.

  • Please remember to close off this discussion by marking Solution for a post that pointed you in the right direction. Or select 'Request split solution' if more than one post helped you.