issue with Dell TL2000 tape library

[tomking]

Hi,

I am not sure if it's something I am doing wrong or Dell TL2000 is simpley a rubbish product. I have so far installed three Dell TL2000 tape libraries. And having same issue with all of them. The first one I did was 2 years ago. It hooked up with a Dell PowerVualt N600 Win Storage 2003 server. My very first 1TB size backup job failed with "0xe00084c8 - The backup storage device has failed." I tried re-install the drive, reboot, re-install, reboot... Eventually Dell came and replace the driver inside TL2000 and since then it works. So at that time I thought the issue was caused by the tape drive hardware...

Ok, 1 month ago, I need to do the same again, TL2000 tape library with Dell PowerVault NX3100 server through a Dell SAS HBA card. And guess what, my first 1TB backup job failed with "0xe00084c8 - The backup storage device has failed. "the detail of the error is like: "The request could not be performed because of an I/O device error. " Exactly the same as the first one I installed 2 years ago!!  I opened a case with Dell and so far no much luck...

OK, when I am waiting response from Dell, I have been assigned to install another TL2000 with another NX3100... identical configuration... and guess what.... identical errors!

I could not get much help from Symantec Support as the error is hardware related! But come one Why I am the only one seems getting problem? Anyone out there got same issue?

BTW, the tape drive in TL2000 is IBM ULT3580-HH5 (as shown in Win devmgmt), I am using the latest Symantec driver for it... The driver firmware is A2C4 (i downloaded and installed yesterday, initially it was sth like A422, maybe I downgrade it instead...), the lib firmware ver is 9.30/3.00e

[CraigV]

Mmm...I take it that your library is on the HCL for your version of BE which you don't mention?
Check to make sure RSM is disabled. If need be, see if Dell have a diagnostic utility and run it against your library with BE's services stopped. If it gives you an error, it is hardware-related.
A further test is to try running an NTbackup job to the drive and see if you get the same error at 1TB!

[JoaoMatos]

Everything happens...
Are you using same cables and terminators?
I had a similar problem that was caused by the cables.

Rgs,
JoaoMatos

[tomking]

Hi Guys,

It turns out Dell so called Pro Support is...m... SH#T!!! It's very unPro support!!!

Why? They spent more than 1 week try to figure out whats wrong and I found something myself, which they should pickup in the first second I mentioned the issue. It's in their DOCO!! I checked Release Note of TL2000... And guess what, under Known Issues, it has these few lines:

•      TL2000/TL4000 – LT05 library drives may fail during the verify portion of the backup job
Job failures may occur during the verify portion of the backup in Symantec BUE 12.5 and 2010.  Although the entire backup job fails, the backup portion does complete successfully and there is no loss of data.  This is due to Symantec not having the appropriate timeout value for the Dell™ PowerVault™ LTO5-140 Tape Drive. 
•        TL2000/TL4000 – LT05 library drives with and 6Gbps SAS HBA receives a controller error
The 6Gbps SAS HBA with firmware version 07.01.24.00 issues a firmware reset for the HBA, which causes Symantec to post a warning “driver detected a controller error on raid port xx”.  This is a recoverable warning in which the drive and HBA resumes operation without intervention. 

So, for anyone out there try to use Symantec Backup with LTO5 tapes, AVOID TL2000 or Dell !!! Unless you can leave backup job UNVERIFIED!

I suppose Symantec will provide a hotfix for the issue soon?

[Larry_Fine]

re: LT05 library drives may fail during the verify portion of the backup job

If your BE 12.5 or BE 2010 has all the LiveUpdates installed, the issue mentioned here is resolved.
http://seer.entsupport.symantec.com/docs/352349.htm

Were your backups failing in the backup phase or the verify phase?

What HBA are you using?  What tape drives were you using in the original problem?
 

[tomking]

Hi Larry,

I have BE 12.5 on one site and 2010 on the other. Both are patched up to date...The job definitely failed during VERIFICATION stage... I had a read 352349... It didn't mention IBM LTO5 tape drive, which is using in Dell PowerVualt TL2000.

The HBA I am using is the one described in Delll doco: 6Gbps SAS HBA controller.

BTW, after test I found the issue does cause restore failure!!! This does not happen for restore small amount of data. But if I want to do a full restore of the backup, it will fail with inconsistant error!

My BE12.5 installed updates: SP4, Hotfix 355944
My BE 2010 installed updates: SP1

Cannot find the hotfix 348315 you mentioned. And LiveUpdate is telling me I am up to date.

[Larry_Fine]

re: I had a read 352349... It didn't mention IBM LTO5 tape drive, which is using in Dell PowerVualt TL2000.

"# Device Timeout values were adjusted to match vendor specifications. "

re: Cannot find the hotfix 348315 you mentioned.

http://seer.entsupport.symantec.com/docs/348315.htm

[tomking]

Hi Larry,

I am going to try this on BE2010. But what about BE12.5? The hotfix is for BE2010 only

[Larry_Fine]

http://seer.entsupport.symantec.com/docs/347420.htm
http://seer.entsupport.symantec.com/docs/350330.htm
# Need to increase timeouts to meet Vendor specifications on LTO drives

[tomking · Accepted solution]

Hi, no luck after manually installed hotfixes. It does not fix my problem. Start to think if it is something with 64bits Win2008... There should be a place I can change SCSI timeout value in Windows...Registry?

[markgibbons81]

Hi there

Any joy with getting to the bottom of this?  I'm having a very similar problem with our Overland NEO2000E tape library and BE 2010R2 U1 (sitting on 2008R2 x64).  The job will run for a random amount of time, and then an eventid 129 (elxstor) will be posted in the system log, and the job will stop writing to tape.  About 20minutes after this the job dies. 

Symantec have no idea, and our tape library support people are also struggling.  It's got to be something to do with the conneciton speeds of the fibre tape devices, but I'm pulling my hair out to figure out what exactly.