20 Steps to Analyze , Correct and Troubleshoot Device issues in Windows

The following 20 steps will help analyze, correct and troubleshoot many different device issues in Window Enviorments.

Just a few Examples (but not limited to )  :

- Status codes : 84 , 96 , 2009

- Drives keep going down or randomly go down.

- No drives available

 

V E R I F Y

1) Update ALL drivers ( tape drives, robot , tape library and HBA)

     -  Nothing should be older than 2010.

     -  ALWAYS obtain drivers from the manufacturer . Symantec has not updated their drivers since 2007

 2) Power cycle everything . (Make sure library is down for at least 3 min)

3) Check and Remove any Phantom Devices -

   From command prompt run :

   set devmgr_show_nonpresent_devices=1

   start devmgmt.msc

   Remove any tape drives, library or robots that are “grayed out”

   REF : http://www.symantec.com/docs/TECH93292

 

4) Install the latest device mappings - http://www.symantec.com/docs/TECH129536

     -  Symantec is constantly updating them so be sure subscribe to the tech note for to receive alerts when a new version is released.

     -  To confirm what version you are on :

        C:\Program Files\VERITAS\Volmgr\bin\tpext -get_dev_mappings_ver

 

5)  Check to see if the drives need cleaning.

     -  Even if you think the drives are clean (by looking at the library’s GUI ) , they must been in a 'clean state' on the Netbackup side.

     -  From the Volmgr\bin\tpclean -L

        If the output contains the message, NEEDS CLEANING you must clean the drive.

 

        You can clean the drive in the admin console or from a command line. Please review the following tech notes:

        http://www.symantec.com/docs/HOWTO43740 - tpclean

        http://www.symantec.com/docs/TECH17537 - How to use NetBackup's drive cleaning utility, tpclean, with a Library

        http://www.symantec.com/docs/TECH37409 - How to set up automatic tape cleaning

 

6) Does the OS see the devices ?

     - Check the OS Device Monitor and make sure they are listed and do not have any yellow exclamation marks.

     - From a command line :  install_path\Volmgr\bin\scan

            * This tells us what the OS sees

            * IF the OS does not see the devices, Netbackup will not see them.  Please contact the hardware manufacturer

7) Does Netbackup see the devices configured ?

     - From a command line :  install_path\Volmgr\bin\tpconfig -d

            * This tells us what Netbackup sees as being configured

8) If using McAfee antivirus , be sure to disable or follow http://www.symantec.com/docs/TECH56658 to make exceptions for Netbackup  processes.  IF this is not done, the most common error you will see is a status 84.

9) For servers that are still running Windows 2003 or earlier, be sure that the ‘Removable storage service’ is stopped and disabled.

 

C O N F I G U R A T I O N

If you have had an issue with drives inconsistently going down, it is sometimes best to delete all devices and reconfigure NOTE : This works best if it is Windows and SSO is NOT being used.

10) CONFIRM NO BACKUPS ARE RUNNING

11) Release all allocations to devices

   From the command line: Install_path\Veritas\NetBackup\bin\admincmd>nbrbutil -resetAll

      (If backups are running, this will cause them to fail. This is why no backups should be running )

12) Delete all devices - Ref http://www.symantec.com/docs/TECH64327

     OPTION 1 – Delete ALL devices.

      From the command line: Install_path\Veritas\NetBackup\bin\admincmd>nbemmcmd -deletealldevices

  OPTION 2 – Delete all devices on a specified media server.

   From the command line: Install_path\Veritas\NetBackup\bin\admincmd>nbemmcmd -deletealldevices -machinename <server name> -machinetype media

13) Use the device configuration wizard to configure your devices.

    - Be sure there are no errors during the wizard or any "unused element" . If so, drag the tape drive to the robot in the wizard.

14) Inventory the robot(s).

 

T R O U B L E S H O O T I N G

15) Look for link state errors on HBA and tape devices

16) Look in the System Event viewer for any hardware related errors including tape alerts.

17) Confirm the following folders have been created in the NetBackup logs directory on the server that contains the devices.

<Install_path>\NetBackup\logs\bptm

<Install_path>\NetBackup\logs\bpbrm

<Install_path>\volmgr\debug\daemon

<Install_path>\volmgr\debug\ltid

<Install_path>\volmgr\debug\reqlib

<Install_path>\volmgr\debug\robots

 

18) To increase logging :

Increase verbosity to 5 in the server host properties -> logging.

Add the line 'VERBOSE' to <Install_path>\volmgr\vm.conf

create the following touch files:

<Install_path>\volmgr\ROBOT_DEBUG

<Install_path>\volmgr\AVRD_DEBUG

<Install_path>\volmgr\DRIVE_DEBUG

<Install_path>\volmgr\SSO_DEBUG

 

19) Run through issue and review logs.  Be sure to change logging back to 0 and remove VERBSOE from vm.conf

20) Information to gather to send to support for troubleshooting :

     - Environment Information (use the server where the devices are located at)

       Server Name :

       OS :

       NBU Version

     - Details for the failed job

     - All logs in #17.

     - NBSU - <NetBackup install path>\NetBackup\bin\support\nbsu -c -t

     - Has anything changes since this issue started occurring ?  A recent upgrade ? Install ?  Hardware replacement?

 

 

Additional Tech Notes

---------------------

http://www.symantec.com/docs/HOWTO33906 - NetBackup media management best practices on Windows

http://www.symantec.com/docs/TECH124594 - Description of Tape Alerts and code definitions

http://www.symantec.com/docs/TECH169477 - Troubleshooting Robot or Drive Issues in NetBackup

http://www.symantec.com/docs/DOC5186  - NetBackup 7.5 Device Configuration Guide

http://www.symantec.com/docs/DOC3656 - NetBackup 7.1 Device Configuration Guide