Unable to restore from tape (TapeAlert code 0x20)

[wired]

Hi,

I'm having an issue with restoring files from tape. I have a couple of different Netbackup 6.5.5 installations on different (yet same hw/sw) systems, but this one particular installation is giving me this error when I try to restore from tape:

<messages about mounting/requesting/granting/positioning A00001>
begin reading
Error bptm (pid=xxxx) cannot read image from media id A00001, drive index 0, Input/output error
Warning bptm (pid=xxxx) TapeAlert Code: 0x20, Type: Warning, Flag: INTERFACE, from drive IBM.ULT3580-TD3.00
Error bpbrm (pid = yyyy) from client nbhost: The following files/folders were not restored:
Error bpbrm (pid = yyyy) from client nbhost: UTF - /dir/file1
Error bpbrm (pid = yyyy) from client nbhost: UTF - /dir/file2
...
restored from image nbhost_zzzzzz; restore time: 0:17:11
Warning bprd (pid=aaaaa) Restore must be resumed prior to first image expiration on Sun 11 Jun 2017 ...
end Restore; elapsed time 0:17:15
the restore failed to recover the requested files (5)

The weird thing is that it says it can back up to tape just fine. Filenames are showing up if I look at the catalog, but I can't restore anything.

Any ideas?

Thanks in advance,
Pedro

[Amaan]

What kind of tape library you have. please have a look into that

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=120&prodSeriesId=...

Upgrading firmware may resolve the issue.

[wired]

It's an IBM tape library. :\

I'm wondering if it's a configuration issue. Like I mentioned, we have other "identical" systems (as far as hardware/software) in which Netbackup is working, but this one is giving me this issue.

[Nicolai]

http://www.symantec.com/docs/HOWTO33004

My guess would be the tape drive internal firmware. Are the all on the same level ?

How are the drives connected - Fibre Channel or SCSI ?

I don't think it a configuration issue.

[mph999]

Nope, Tapealerts are sent from the drive, and are hardware related issues.

This is what it means.

0x20: 'Problem with SCSI interface between tape drive and initiator',

The initiator is the HBA (far as I know) so, there is some problem between (and possibly including) the HBA - SAN - Drive.

For example

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c02199767&jumpi...

This was a firmware issue

You will probaby need to talk with IBM.  FYI, NBU cannot cause TapeAlerts.

Martin

[Yogesh9881]

Lets start from "zero" 

1. How many media server in this NBU domain ?

2. what NBU version on Client & what is OS ?

3. Is it Windows restore or UNIX or database ?

5. Single Library or multiple ?

4. How many tape drives ? is it SSO ? LTO4 or LTO5 or LTO2 ?

5. Is your IBM tape drives drivers updated at the recent version ?

6. at the time your restoration fails is there any OS logs created ?

7. Is there any logs at library level at the time of restore failed ?

8. the data which you are trying to restore, is that media is imported in this NBU domain or existing media ? 

[mph999]

Not sure we need all this in this case Yogesh.  As you wil know, I am always hoping people wil provide plenty of information, but in the case of Tapealert it is very very simple - it is not NBU.

So I think thetype of job, config etc ... is not important.

It is impossible for NBU to cause a Tapealert .... (and this was confirmed to me by oe of the top Tape drive guys in the US).

We see the meaning ;

0x20: 'Problem with SCSI interface between tape drive and initiator',

So the problem is between the HBA and drive, this is 'after' NBU and even after the 'tape drive driver'.

I think we could desribe the different bits like this ...

NBU | OS | DRIVERS | 'SCSI STUFF IN OS' |HBA FIRMWARE| HBA | 'SAN STUFF' |DRIVE FIRMWARE | TAPEDRIVE

So, the problem is somewhere in these components:

 HBA FIRMWARE| HBA | 'SAN STUFF' |DRIVE FIRMWARE | TAPEDRIVE

Martin

[Yogesh9881]

Thnakx for the update.

Actually i posted above questions & then i saw your reply about "0x20: 'Problem ".

[wired]

Awesome, thanks for the replies, guys. It makes sense that it isn't NBU, since like I said, we have other configurations, all "the same", yet this is the only one with a problem. I'll pass on this information, and report back any findings.

Thanks again,
Pedro

edit: after re-reading this information, I still have a question that may or may not be relevant. NBU is reporting success on backup operations, even though restore doesn't work. What do you guys think that means? I wonder if backups are indeed completing successfully, or if they're also failing somehow. I guess I don't know how NBU/tape drives work, but I would think that if one operation doesn't work, the other one won't work, either. 

[mph999]

Hey Yogesh, Ahh that explains it - I thought it was odd that you were asking ....

[Andy_Welburn]

How much time elapsed between the two?

Backup -----> something 'broke' -----> Restore

***EDIT***

Or is it immediate backup -> restore attempt

****EDIT #2****

They are different operations tho' read & write!

My car works, it drives forwards but not backwards - very poor analogy I know but then I'm not a tape drive expert (nor a car mechanic either!)

[wired]

I'm not sure this question is in the scope of what we're talking about here, but is it the device drivers that I need to upgrade, or the firmware? 

The reason I ask, is because I looked up the Veritas NetBackup Enterprise Server and Server 6.5 Hardware Compatibility List, and from what I see, my firmware version is supported. Here is the info from my system:

Drive:
Product ID: ULT3580-TD3
Firmware Version: 73P5  (compatibility list says that 4B51, 51TB, 54K1, 57F7, 59D0, 59D2, 64A0, 64D0, 69U2 were tested)

Library:
Product ID: 3573-TL
Firmware: 3.70/2.30e (compatibility list says 1.00, 1.70 and 4.40 were tested)
Bootcode Firmware Revision: 0.60

So, it seems I'm good with firmware.

Now, IBM's Setup, Operator, and Service Guide for the TS3200 tape drive says:

Device Driver Installation:

Ensure that the proper device driver, if applicable, is installed for the library.

Note: Many backup applications use their own drivers for the library and drive.

Before installing a driver, make sure it will not be in conflict with the

software. Contact your Backup Application vendor for this information.

I tried using rpm -qa and "locate" to find lin_tape and IBM tape drivers on the machines and didn't find anything. Like I mentioned, we have other installations with this same configuration, and those work. So I'm officially stumped at what could be the problem. Should I go ahead and try installing the lin_tape drivers? Are they needed at all? What do y'all think?

Thanks again guys, I appreciate your help.

Pedro

[mph999]

No idea I am afraid, from my previous post

0x20: 'Problem with SCSI interface between tape drive and initiator',

So the problem is between the HBA and drive, this is 'after' NBU and even after the 'tape drive driver'.

I think we could desribe the different bits like this ...

NBU | OS | DRIVERS | 'SCSI STUFF IN OS' |HBA FIRMWARE| HBA | 'SAN STUFF' |DRIVE FIRMWARE | TAPEDRIVE

So, the problem is somewhere in these components:

 HBA FIRMWARE| HBA | 'SAN STUFF' |DRIVE FIRMWARE | TAPEDRIVE

So the issue could be anywhere, gbic fault, port fault on switch, drive fault ...

Martin

[mph999]

"NBU is reporting success on backup operations, even though restore doesn't work. What do you guys think that means?"

This is quite easy to explain.

NBU sends the data to the OS.  The OS sends it, via the tape driver to the tape drive.

Any issues therefore, happen outside NBU and 'we' can't see them.  If there is some issue, eg firmware, its quite possible that the 'fault'/ bad firmware causes the tape drive to write bad stuff, but not send out any error, so we think it is all good.

Martin