cancel
Showing results for 
Search instead for 
Did you mean: 
Dyneshia
Level 6
Employee

The following 20 steps will help analyze, correct and troubleshoot many different device issues in Window Enviorments.

Just a few Examples (but not limited to )  :

- Status codes : 84 , 96 , 2009

- Drives keep going down or randomly go down.

- No drives available

 

V E R I F Y

1) Update ALL drivers ( tape drives, robot , tape library and HBA)

     -  Nothing should be older than 2010.

     -  ALWAYS obtain drivers from the manufacturer . Symantec has not updated their drivers since 2007

 2) Power cycle everything . (Make sure library is down for at least 3 min)

3) Check and Remove any Phantom Devices -

   From command prompt run :

   set devmgr_show_nonpresent_devices=1

   start devmgmt.msc

   Remove any tape drives, library or robots that are “grayed out”

   REF : http://www.symantec.com/docs/TECH93292

 

4) Install the latest device mappings - http://www.symantec.com/docs/TECH129536

     -  Symantec is constantly updating them so be sure subscribe to the tech note for to receive alerts when a new version is released.

     -  To confirm what version you are on :

        C:\Program Files\VERITAS\Volmgr\bin\tpext -get_dev_mappings_ver

 

5)  Check to see if the drives need cleaning.

     -  Even if you think the drives are clean (by looking at the library’s GUI ) , they must been in a 'clean state' on the Netbackup side.

     -  From the Volmgr\bin\tpclean -L

        If the output contains the message, NEEDS CLEANING you must clean the drive.

 

        You can clean the drive in the admin console or from a command line. Please review the following tech notes:

        http://www.symantec.com/docs/HOWTO43740 - tpclean

        http://www.symantec.com/docs/TECH17537 - How to use NetBackup's drive cleaning utility, tpclean, with a Library

        http://www.symantec.com/docs/TECH37409 - How to set up automatic tape cleaning

 

6) Does the OS see the devices ?

     - Check the OS Device Monitor and make sure they are listed and do not have any yellow exclamation marks.

     - From a command line :  install_path\Volmgr\bin\scan

            * This tells us what the OS sees

            * IF the OS does not see the devices, Netbackup will not see them.  Please contact the hardware manufacturer

7) Does Netbackup see the devices configured ?

     - From a command line :  install_path\Volmgr\bin\tpconfig -d

            * This tells us what Netbackup sees as being configured

8) If using McAfee antivirus , be sure to disable or follow http://www.symantec.com/docs/TECH56658 to make exceptions for Netbackup  processes.  IF this is not done, the most common error you will see is a status 84.

9) For servers that are still running Windows 2003 or earlier, be sure that the ‘Removable storage service’ is stopped and disabled.

 

C O N F I G U R A T I O N

If you have had an issue with drives inconsistently going down, it is sometimes best to delete all devices and reconfigure NOTE : This works best if it is Windows and SSO is NOT being used.

10) CONFIRM NO BACKUPS ARE RUNNING

11) Release all allocations to devices

   From the command line: Install_path\Veritas\NetBackup\bin\admincmd>nbrbutil -resetAll

      (If backups are running, this will cause them to fail. This is why no backups should be running )

12) Delete all devices - Ref http://www.symantec.com/docs/TECH64327

     OPTION 1 – Delete ALL devices.

      From the command line: Install_path\Veritas\NetBackup\bin\admincmd>nbemmcmd -deletealldevices

  OPTION 2 – Delete all devices on a specified media server.

   From the command line: Install_path\Veritas\NetBackup\bin\admincmd>nbemmcmd -deletealldevices -machinename <server name> -machinetype media

13) Use the device configuration wizard to configure your devices.

    - Be sure there are no errors during the wizard or any "unused element" . If so, drag the tape drive to the robot in the wizard.

14) Inventory the robot(s).

 

T R O U B L E S H O O T I N G

15) Look for link state errors on HBA and tape devices

16) Look in the System Event viewer for any hardware related errors including tape alerts.

17) Confirm the following folders have been created in the NetBackup logs directory on the server that contains the devices.

<Install_path>\NetBackup\logs\bptm

<Install_path>\NetBackup\logs\bpbrm

<Install_path>\volmgr\debug\daemon

<Install_path>\volmgr\debug\ltid

<Install_path>\volmgr\debug\reqlib

<Install_path>\volmgr\debug\robots

 

18) To increase logging :

Increase verbosity to 5 in the server host properties -> logging.

Add the line 'VERBOSE' to <Install_path>\volmgr\vm.conf

create the following touch files:

<Install_path>\volmgr\ROBOT_DEBUG

<Install_path>\volmgr\AVRD_DEBUG

<Install_path>\volmgr\DRIVE_DEBUG

<Install_path>\volmgr\SSO_DEBUG

 

19) Run through issue and review logs.  Be sure to change logging back to 0 and remove VERBSOE from vm.conf

20) Information to gather to send to support for troubleshooting :

     - Environment Information (use the server where the devices are located at)

       Server Name :

       OS :

       NBU Version

     - Details for the failed job

     - All logs in #17.

     - NBSU - <NetBackup install path>\NetBackup\bin\support\nbsu -c -t

     - Has anything changes since this issue started occurring ?  A recent upgrade ? Install ?  Hardware replacement?

 

 

Additional Tech Notes

---------------------

http://www.symantec.com/docs/HOWTO33906 - NetBackup media management best practices on Windows

http://www.symantec.com/docs/TECH124594 - Description of Tape Alerts and code definitions

http://www.symantec.com/docs/TECH169477 - Troubleshooting Robot or Drive Issues in NetBackup

http://www.symantec.com/docs/DOC5186  - NetBackup 7.5 Device Configuration Guide

http://www.symantec.com/docs/DOC3656 - NetBackup 7.1 Device Configuration Guide

Comments
jason_qb
Level 3

very good! just a driver down question ,study. create the following touch files:(good) \volmgr\ROBOT_DEBUG \volmgr\AVRD_DEBUG \volmgr\DRIVE_DEBUG \volmgr\SSO_DEBUG do you have same tools that analysis of the logs?

Dyneshia
Level 6
Employee

Hi Jason, are you asking what you should look for in those logs ?

jason_qb
Level 3

yes, would you  give me some suggest?

tks,Dyneshia.

Dyneshia
Level 6
Employee

In most logs, you can look for :

<16>

<32>

fail

error

ERR

WRN

cannot

Here are some "compelte" common errors :

 

<Install_path>\volmgr\debug\daemon

<16> peer_hostname: gethostbyaddr() h_errno: (11004)

 


<Install_path>\volmgr\debug\ltid


<16> emmlib_SetRobotStatus: (0) UpdateLibrary failed, emmError = 2007087, nbError = 0

<16> emmlib_UpdateMachineState: (0) CORBA call threw exception.

<16> SendEmmHeartbeat: (-) Translating EMM_ERROR_CorbaException(3000004) to 334 in the device management context

 <16> WriteEntry: (-) Translating EMM_ERROR_DriveAlreadyAllocated(2005004) to 258 in the device management context


<Install_path>\volmgr\debug\robots

<6> MultipleEjectLoop: eject aborted, exit_status = 44

<5> MultipleEjectLoop: eject_aborted

<3> move_medium: Invalid operation in move_medium(), op = 24

Dyneshia
Level 6
Employee

If the article helped you be sure to vote it up :)

srvz_nik
Level 1

very good post yes, would you please write a public technote as well just as this one .

 

Dyneshia
Level 6
Employee

Definitely  :)  Its been on my to do list.

mydhling
Not applicable

so  good ! I  met a difficult about my Netbackup on media server recently.But I cann't solve it by my effort.I describe the problem following:

 

my operation system is windows 2003  on media server. my  tapes is on EMC DD670.I  give two drives to media server(media server name is qzmail).The hardware scan for the drives on operation system is right.But  in the c:\Program Files\Verteris\Volmgr\bin\tpconfig -d ,the drives are have many  paths.dhl1.png    Then,I use the command like c:\Program Files\Vertris\Volmgr\bin\tpconfig -delete -drpath -path {1,0,0,14}  -asciimame IBM.ULIRIUM-TD.043 to delete the wrong path. then tpconfig -d ,the drives  and the paths are right ,but scan.exe  show that  the wrong path are not delete. The problem is how to produce?  And   how to solve it.I am so crazy with it and have no idea about it.dhl2.png            

 

 

 

 
Dyneshia
Level 6
Employee

It seems like you are using SSO ?  Please move topic to the forum.  I tried to do it for you, however could not.

 

https://www-secure.symantec.com/connect/backup-and-recovery/forums/netbackup

Dyneshia
Level 6
Employee

Just created :

http://www.symantec.com/docs/TECH205200

Should be published in a week or less.

mydhling
Not applicable

Yes,I am using SSO.Thank you for your help.Now,I solved  the problem by rebooting the operating system,then the resault of scan  is right.Maybe something temp files of theoperating system  need to  eliminate.Though it is difficult for the  production system,sometime rebooting is a very good method to solve problems.

rajeshthink
Level 4

Aesome Post dyneshia , 1 stop shop for tape drive issue fix , but we also need ACSLS issues to posted in.

 

cheer,

Rajesh Kumar

 

 

david_raj
Level 3

how to troubleshoot the drive down issue?

 

 

Version history
Last update:
‎02-10-2013 12:22 PM
Updated by: