Media is unrecognized error even on erased tapes

[RichC1]

We recently replaced a failed LTO-3 Tape Library with a new LTO-4 library to use with Backup Exec 2010 R3 (yes, I know it's old, no I can't upgrade). They are the same model of library, the only difference is the LTO-4 drive instead of LTO-3. It is on the HCL for BackupExec 2010. 

The library passes diagnostics successfully using the vendor diagnostic app. However, Backup Exec cannot use any of our LTO 3 tapes anymore, always giving an error of "Media is unrecognized. Erase the media to make it useable". I have tried a quick erase and a long erase and it still gives the same error. I have tried two full sets of tapes (30 tapes total). 

I have tried both the quantum and the symantec drivers, it makes no difference. 

I contacted the tape library vendor (quantum) and they said based on the diagnostic logs it is a problem with the backup software.

Any ideas how I can get this working? 

[Larry_Fine]

What tape library?

What interface?  SAS?  FC? 

What HBA on the server?

Have you tried any brand new LTO3 or LTO4 tape cartridges?

[RichC1]

Quantum Superloader 3, SCSI interface 

The Xtalk diagnostic tool from Quantum has no problem reading and writing to the tapes. I've had it do an 18GB write and read test and it passes with flying colors. Even shows up in the library in BE as having 18GB used on it when inventoried. 

The SCSI read/write diagnostic also passes without error.

Again, this all worked fine with our previous Superloader 3 which was also SCSI but was LTO-3. The only thing that changed was swapping out the SL3 with LTO 3 for an SL3 with LTO 4. 

[Riyaj_S]

Capturing Device and Media service verbose logs would help in this case:

https://www.veritas.com/support/en_US/article.TECH63629

Thanks and Regards,

Riyaj

[RichC1]

Verbose logs attached

[Riyaj_S]

PVLSVR:   [08/04/17 08:12:10] [3960]     AdammSession::IsOverwriteMediaMounted() - ERROR = 0x000006F9 (ERROR_UNRECOGNIZED_MEDIA)

Tape drive is not able to recognize media. This could happen if there is any issue with the tape device driver or incompatible media is used. Re-installation/updating of tape device driver or using a completely fresh compatible media may help?

Thanks and Regards,

Riyaj

[RichC1]

I've tried 30 tapes so far. I've tried both the symantec and quantum device drivers, including completely uninstalling and reinstalling them.

Further, the tape device does work fine and can read and write the media, I was able to have it do a full diagnostic including a read and write test using Quantum's xtalk diagnostic software; using that software, the tape drive could mount, read, write the tapes without issue. It did an 18GB write and read test without error.

The error ONLY happens in Backup Exec, so it seems clear it isn't the tape drive nor is it the drivers, it has to be something with the backup software. Any ideas what it could be? 

[Larry_Fine]

What was the failure of your LTO3 SuperLoader3?

What specific SCSI HBA are you using in your server?

Have you tried any brand new LTO3 or LTO4 tape cartridges?

[RichC1]

The SL3 failed to power on one day. We had been using that for years without an issue prior. I suspect the power supply gave out, as the fans don't even turn when power is applied.
It's a Dell 5/i IIRC. 

I don't have any LTO-4 tapes, and we don't have budget to replace our LTO-3 with LTO-4. This drive should work with LTO-3 just fine. 

Again, I can get the new tape drive to read and write these same tapes with software other than backup exec. I already went to Quantum for hardware support first. Quantum examined the diagnostic logs and ensure the drivers and firmware were correct. Quantum Advanced Support reviewed the hardware logas and they indicated the problem is with Backup Exec and that the drive and loader are functioning properly. 

Looking for what could cause this from a backup exec perspective. 

[Larry_Fine]

I think the 5/i is a SAS HBA?  The reason for the question is to make sure that you were NOT using a RAID HBA as most RAID HBAs are not supported for tape.  See http://www.veritas.com/docs/000038683  Using a RAID HBA can cause very weird issues, which may be triggered by having a different & faster library attached.  Both your LTO3 and LTO4 libraries are parallel SCSI, right?

I assume that your Backup Exec quick erase and long erase jobs are labeled as successful?

Can you post your adamm.log file?

Can you post a job log (in mht format) of an inventory of a single slot, along with a SCSI trace of that time frame (in .bin format)  https://www.veritas.com/support/en_US/article.000015698

I have a fear that something was wrong with your original LTO3 drive such that the tapes would not have been readable on any other drive.  Something such as a bad tape head or a misaligned head?  This is speculation on my part.

I suspect that the Quantum diag tool is reading/writing in a different method than BE uses.  Can the Quantum diag software read your old tapes written by Backup Exec?  or must it do a write test before it does a read test?  I fear that what Quantum is testing is only that it can read test data that it just wrote, again speculation, but trying to help.

I would really be curious to see if BE works with a brand new tape, either LTO3 or LTO4.  If BE works with the new tape cartridge, then that somehow implies your existing tapes are "damaged".  Unfortunately, it means they are not restorable and it means you may need to purchase new tapes.

You may want to investigate repairing or replacing (even temporarily) the power supply in your LTO3 library to see if that fixes the problem.  That gives you another device to test the readability of your old LTO3 cartridges.  And replacing the power supply may be less costly than immediatly buying a large batch of new tape cartridges.  Then you can purchases new tapes in smaller quantities as you have budget.

[RichC1]

It's SCSI, not SAS. We've been using this HBA for many years, and until the LTO-3 one died, it was working perfectly. 

The Erase jobs were successful. An inventory is one of the few things that doesn't trigger this error, by the way; I can inventory tapes no problem without getting the error. 

Logs are attached. 

If it were my decision, I would have replaced the power supply on the LTO-3 library already. It is not my decision. 

[Larry_Fine]

Since the inventory job works OK, that implies that the tape is readable by BE.  Maybe you have some sort of a corrupt Backup Exec database or catalog?

I believe there is a process with BEUTILITY to rename your current database/config and start clean (essentially acting as a new install of BE).  Unless you happen to have a spare computer that you could do a fresh isntall of BE on as a test?

[RichC1]

That sounds promising. Any idea where I can find the procedure for starting over with a new database and catalog? As long as I could backup/archive the old one just in case, I'm okay with scrapping the current one. 

[Larry_Fine]

I haven't done it in many years and I bet things have changed.  I did some searching for BEUTILITY, but I didn't find what I was looking for, so hopefully somebody more experienced in this area will chime in.

[RichC1]

I dumped the database then did a "reload from base" to start with a fresh database as per this article: https://www.veritas.com/support/en_US/article.TECH66780 .

When I inventoried the tapes for the first time, they showed up as "Retired Media". Moving them to Scratch Media and trying a backup operation results in the same Media is Unrecognized error. A subsequent inventory keeps them in scratch media, but still no dice otherwise. 

[RichC1]

Still looking for some ideas to troubleshoot this issue. Any help is appreciated. 

[RichC1]

I tried a brand new tape and received the same error.