Tape Drive Going OffiIne

[revenge_of_the_]

Hi Everybody

I have a HP Proliant ML350 G6 with 2003 R2 (64) and a HP DAT160 USB tape drive with HP drivers and Backup Exec 2010R3 with all the updates applied.

After restarting the Backup Exec services I can run one backup but that night the next back up will fail with the tape going offline.  I have checked the event logs and there is no entry for events 5/7/9/11/15.  I have also disabeled the Removable Storage service.

I have also run the HP tape drive check utility (HP L&TT) which updated the firm ware but does not indicate any issues with the hardware.

Any ideas please?

Thanks

[Larry_Fine]

I would assume that is a clerical error or an oversight on HP's part.  It makes no sense for them to skip a version since they support the version before and after BE 2010.

Tracer is proibably your next step, but I fear it will just show you what the adamm.log shows.  The implications are that this is not a BE issue.

[revenge_of_the_]

Is it possible a lack of available memory could cause this?

[CraigV]

How much memory do you have installed, and how are the resources being used during the backup?

Thanks!

[revenge_of_the_]

4GB memory installed

Task Manager>Performance shows:

Physical Memory (k):

Total: 4183654

Available: 743208

System Cache: 1079484

Page File set to 6127MB and 4.11GB of Page File in use according to Task Manager>Performance.

So about 750MB of memory available.

[CraigV]

...that should be good-enough. Run a backup job and do a perfmon at the same time and see what happens when the remote agent service stops...

Thanks!

[revenge_of_the_]

Nothing interesting in process manager.  I had to restart the Backup Exec service to get it to run.  It always runs fine if i restart the services and run it.  But if i leave it to run overnigth it fails.

I tried changing the start time from 1am to 4:30am in case something else was running but no difference.

Getting exactly the same issue on another server now.  Only 6 weeks old ML 350G8, Windows 2008R2, a LTO tapedrive and BackupExec 2012.  It started a couple of weeks ago but I thought i had fixed it by updating the firmware with HP.

It had worked for over a week but it has now gone again.

So different server Generation, different type of tape drive, different OS and different version of Backup Exec but EXACTLY the same failure.

Only similarity is they both run Oracle.  Can't believe this would make a diference though.

[Larry_Fine]

Getting exactly the same issue on another server now.  Only 6 weeks old ML 350G8, Windows 2008R2, a LTO tapedrive and BackupExec 2012.

What interface to the LTO tape drive?

What HBA?  Make sure it is a supported or non-raid HBA.  http://www.symantec.com/docs/TECH70907

[revenge_of_the_]

Looks like they have a HP H222 Host Bus Adapter in Slot 1 with a HP ULTRIUM920 DRV plugged into it.

The RAID is running off a Smart Array P420i Controller in Slot 0

Thanks

[CraigV]

...that card is a supported card for external tape drives...

http://h18004.www1.hp.com/products/quickspecs/14337_na/14337_na.pdf

Thanks!

[revenge_of_the_]

So any ideas where I go next?

HP are taking out a new USB drive for the first server today and fitting it even though it is not showing any errors..

[CraigV]

Hi,

I'd suggest doing the following:

1. Upgrade the ProLiant Support Pack on that ML to the latest firmware...across the board.

2. Log a call with Symantec to check this out.

Thanks!

[Netwest]

Any progress on this.

We have the same probem

DL380 G8 H222 BE 2010 2008R2

H222  Latest Firmware

          Latest Windows driver

HP Ultriul 3000 LTO5 Latest Firmware

Tried both HP and Symantec tape drive windows drivers.

[20180] 01/03/14 11:20:18.328 DeviceIo: 03:00:03:00 - Device error 1117 on "\\.\Tape0", SCSI cmd 34, 1 total errors

[20180] 01/03/14 11:20:23.368 PvlDrive::DisableAccess() - ReserveDevice failed, offline device

       Drive = 1003 "HP 0001"

       ERROR = 0x0000001F (ERROR_GEN_FAILURE)

[20180] 01/03/14 11:20:23.407 PvlDrive::UpdateOnlineState()

       Drive = 1003 "HP 0001"

       ERROR = The device is offline!

Windows Event Log shows LSI-SAS2 ID 11 The driver detected a controller error

CDrom device also show the same error within seconds in Event log.

Looks like the tape drive can be access as a CDROM from within windows but none is visible.

This looks like a timing error on the SAS but no resolution.

Any ideas???

[Moe_Howard]

@Netwest: You are having a different problem based on this comment, "Windows Event Log shows LSI-SAS2 ID 11 The driver detected a controller error"  The problem should go away after resolving the problem with the SAS2 HBA.

[Netwest]

After reviewing the adamm.log for some time I realized the tape device was randomly being detected as tape0 or tape1. Whenever tape1 was detected BE would complain and take the device Offline.

I'm not sure how often BE runs the Device Discovery process but the device would randomly go Offline.

I have re-run the device configuration  wizard and selected the option to remove unused devices. It was previously showing 2 other unused device during this prcess and now only shows the current device.

This machine has had a number of tape drive changes due to problems with a new LTO tape causing the drive loading leader to dislodge, require a drive replacement. It took 3 replacement drives before we discovered the problem. As at least one of these previous drives were on a different SCSI id, there were registry entries for unused devices that may have been detected by BE.

Now waitng to see if the problem is resolved..

[CraigV]

...running that particular option should be done whenever you replace a drive.

Thanks!

[Netwest]

Not clearly documented, particularly deleting old drivers, and clearly causing problems with BE device discovery process. This can be seen in Revenge of the Cream's adamm.doc where tape0 is sometimes detected as tape1 and is failing Offline. I have seen quite a number of "device offline" posts in the forums without resolution. These likely to be a victim of this problem. 

[Moe_Howard]

It's concerning that the tape drive appears as Tape0 or Tape1 at times, in the OS. If there is only one tape drive attached to the server then it should enumerated as Tape0.

When the tape drive is reporting as Tape1, look at the following key to see what may be listed as Tape1:

\\HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\Scsi

Click on the Scsi hive then press the asterisk key to expand all the Scsi ports and start looking for anythign at Tape0. Maybe this will help expose the root cause of the problem.

[revenge_of_the_ · Accepted solution]

Hi

Finally got a solution:

On the media server, we changed the user mode drivers to kernel mode drivers.For this we set DioDrivers from 2 to 0 in HKLM\SOFTWARE\Symantec\Backup Exec For Windows\Adamm and we restarted the BE services.

Been working OK for a few weeks now.

Thanks to all