09-11-2014 03:22 AM
Hello,
I've got one shared tape library which is used by Master Server and few Media Servers.
Media servers has no problems with backup jobs, but backup jobs from master servers stops in Mounting / Connecting state.
All policies uses the same tapes and drives, so I think that problem is only by master server side.
Could anyone tell me, how can I troubleshoot problem with my Master Server, what logs should I check, etc.
Thank You in advance
Solved! Go to Solution.
09-12-2014 04:56 AM
From your logs the drive will not get used until it has been cleaned .. get a clenaing tape, clean it and see if it then works:
12:28:16.356 [6368.1980] <16> init_tape: GetTapeStatus() on \\.\Tape1 (SCSI coordinates {5,0,1,0}), failed, The device has indicated that cleaning is required before further operations are attempted. (1165)
12:28:16.356 [6368.1980] <16> io_open: cannot open file C:\Program Files\Veritas\NetBackup\db\media\tpreq\drive_IBM.ULT3580-HH4.002, The device has indicated that cleaning is required before further operations are attempted. (1165)
09-11-2014 03:32 AM
Create bptm log folder under ..\netbackup\logs.
Also add VERBOSE entry to ..\volmgr\vm.conf and restart NBU Device Management Services.
NBU operations will be logged in bptm log.
Media Manager / hardware actions and errors will be logged in Event Viewer System and Application logs.
Which Windows version on the master?
We can provide additional info based on the version.
09-11-2014 03:58 AM
OS version is 2008 R2 SP1.
Bptm log shows:
process_tapealert: TapeAlert returned 0x00000000 0x00000000 (from tapealert_and_taperelease)
tapealert_and_release: report_attr, fl1 0x00100001, fl2 0x00000000
Restart NBU Device Manager Service didn't logs any errors.
Event viewer shows error about old robot number from one of media servers (some time ago I changed tape library, but after this backups works fine for master server).
09-11-2014 05:03 AM
Update:
When I changed backup destination for one tape for one of problematic policy backups started normally. It looks that master server has problem with connection to one drive.
09-11-2014 05:40 AM
Please copy bptm log to bptm.txt and upload as File Attachment.
I would like to compare successful backup with previous unsuccessful attempt.
About Event Viewer logs:
Have you added
VERBOSE
to vm.conf on the master before restarting Device Management Service?
09-11-2014 08:30 AM
This statement indicates you probably have an issue with Ghost devices on your Windows Master Server:
"Event viewer shows error about old robot number from one of media servers (some time ago I changed tape library, but after this backups works fine for master server)."
So, a little understanding of how Netbackup works is required here :)
Netbackup is incapable of scanning hardware. It uses an API to call the operating system function to scan the hardware and then the OS 'presents' the available hardware to NBU.
Windows has a known issue of 'ghost devices' so anytime you change a drive or robot on a Windows Master/Media server the OS continues to 'present' this non-existant hardware to Netbackup who is fat dumb an happily allocating phantom hardware resources.
The following technote explains how to remove these ghost devices:
http://www.symantec.com/docs/TECH93292
09-12-2014 04:32 AM
I added yesterday VERBOSE to vm.conf and after restarting of Device Manager Service nothing is loged to the bptm.log file. I deleted Ghost devices jmontagu way (there was two ghosted medium changer and two tape drives). I also deleted storage unit for Master server and one Media server and ran Storage Devices Wizard. Right now Event viewer doesn't displays error about old robot number for this media server, so something goes forward. Main problem remains. In bptm.txt You can find bptm log for successful backup to one tape (X658L4) and unsuccessful backup to two tapes (started at 13:06). Detailed_Status.txt file contains logs from Activity Monitor.
09-12-2014 04:56 AM
From your logs the drive will not get used until it has been cleaned .. get a clenaing tape, clean it and see if it then works:
12:28:16.356 [6368.1980] <16> init_tape: GetTapeStatus() on \\.\Tape1 (SCSI coordinates {5,0,1,0}), failed, The device has indicated that cleaning is required before further operations are attempted. (1165)
12:28:16.356 [6368.1980] <16> io_open: cannot open file C:\Program Files\Veritas\NetBackup\db\media\tpreq\drive_IBM.ULT3580-HH4.002, The device has indicated that cleaning is required before further operations are attempted. (1165)
09-12-2014 05:18 AM
I know that it should be cleaned, but right now it is impossible. If it is the reason, tell me why Media Server with Linux OS is able to use both drives? If on Linux media servers Tape alert is off and it doesn't know about cleaning or should I check something else?
09-12-2014 06:37 AM
I assume it is to do with the paths but not sure why that should be the case unless it goes back to the old drives you had
I am unsure why cleaning is impossible? The alert comes from the drive firmware and should not really be ignored so best to just clean it and see if that resolves your issue.
09-12-2014 07:22 AM
I know that it should be cleaned, but right now it is impossible.
Why is cleaning impossible?
I agree with Mark - the TapeAlert is coming from the drive firmware - nothing to do with the master/media server or NetBackup.