NetBackup "block read is not a NetBackup or BackupExec media header,172"

[kronicker_chief]

Hello respected sirs,

I have a virtual environment consisted of a virtual tape library hosting 8 robots and 2 virtual tape drives, a Windows 2008 r2 master server, and 2 medias, both Win Server 2008 r2 x64, and a Centos 6.2 x64 as a client. Master, medias and the client have NetBackup 7.5.

So far so good, but Duplication job, either selected through a SLP either manually on a basicdisk from Catalog ends up with "Error bptm block read is not a NetBackup or BackupExec media hearder , len=1024 , media ID 1018L4, drive index 2 , data is unknown", and later in same job details "read failed, cannot read media header, may not be NetBackup media or is corrupted ( 172)". Same for synthetic backup jobs, when Netbackup starts to read some tapes in order to compile the synthetic backup.

I have already expired and deleted once all the tapes, deleted the robots and drives configuration, restarted VTL, iscsi's, re-configured robots and media, and it helped for a while.

Any help would be mostly appreciated, thank you !

[Nicolai]

What happens if you run a media verify:

Netbackup GUI -> Netbackup Mangement -> Catalog - Verify 

The other 8 robot - do they contain Netbackup virtual tapes also or are they in a "alien" format ?

[kronicker_chief]

Hello,

The verify command does as it should : it displays a list of backup id's.

All tapes appear configured, put to different Volume Pools, i can vmquery or bpmedialist, i can freeze / unfreeze, them , i do not see anything unusual in the robots / tapes configuration.

Sometimes even in a succesful backup, Netbackup tries some tapes, bptm goes into the same error, job details retrieve "FREEZING <media_id>".. and after a couple of tries and freezes, finally a "good " tape is found, and the backup is executed.

Thank you !

[sksujeet]

Were you using these medias on different Backup software or this was alwasy been used with NBU.

job details retrieve "FREEZING <media_id>".. and after a couple of tries and freezes, finally a "good " tape is found, and the backup is executed.

These medias which are freezing seems to have some issue. Either they were not used with NBU before or were written by some other backup software. Just label these tapes again make sure uncheck the verify option while doing so

Note: labeling will erase all the data from tape.

[Nicolai]

What is the syslog/error messages say on the VTL ?

Either media has been overwritten by other systems using the same logial robot

or

There is a hardware issues with the VTL.

A "Error bptm block read is not a NetBackup or BackupExec media hearder , len=1024" should not show up  under normal operation.

[kronicker_chief]

@sazz,

tapes have been always used by NBU

even if I uncheck the verify option or i run it through command, labelling ends up in error "error occured verifying label", but same error even when i label a tape that hasn't been involved in any backup / duplication

@nicolai,

the syslog/error doesn't display any issues

it's highly unlikely that tapes are written by other systems, i have stated the environment's configuration

-i didn't see these issues until i have started to make SLP's, duplications, advanced and basic disks.

Thank you for your help

[kronicker_chief]

I have found some STKLIB errors in media and master system log, where STK is one of the 2 types of tape drives. 

I think i will interrupt iscsi connections, uninstall and reinstall tape drives and medium chnager drivers, re-delete and make again tape drives and robots configuration

I'll keep u up to date, thank you very much.

[Mark_Solutions]

What data buffer size do you use? just wondering if it just cant read it

[Nicolai]

Don't think that will work. If you are using iSCSI you are having a network issues and can explain the errors you are seeing.

[kronicker_chief]

I have reinstalled the drivers, deleted the devices configuration, a catalog backup works fine, a duplication ends up again in same error

I am using an unchanged 65536 bytes data buffer size, and

"info bptm setting receive network buffer to 236168 bytes"

@Nicolai, the iscsi seems to work fine, the drives detection is OK all devices are detected , attached to media servers and configured. Again, I am into a virtual environment. 

Maybe you could use some logs. What log can I attach?

[Nicolai]

bptm - debug level 5

Please let the network admin verify if switch port report CRC errors.

[kronicker_chief]

I have attached also the duplication job details.

Thank you !

[Mark_Solutions]

Interesting line:

10:11:09.936 [4936.2644] <16> get_mediaDB_byid: media ID 3005TA found; but it is assigned to win2k8med.test.lab and not win2k8mst.test.lab
 

Also seems to be a lot of encryption discussion going on - have you ever used encryption on these tapes - or are you using it?

It does sound like there is something amiss - have you tried to do a bplabel or an erase on a tape before using it? This will set its header up as required.

[Nicolai]

1001L4 & 1013L4

Seems to fail repately - label them or move the media to the none pool. 

You are using  LinuxVTL - This pice of software (great or not)  is not on Symantec Hardware Compatibility List

[kronicker_chief]

Mark,bunfortunately i see that labelling and erasing media ends up in error 84, although i uncheck the "verify label" option

I haven't used encryption, but mixed up a Linux client, backing up succesfully files from it.

I have un-shared the robots, leaving them attached only to the master and a media servers, same error.

At the line 10:11:09.936 [4936.2644] in bptm, i was trying to deassign all medias in order to delete and re-add them in NetBackup.

Nilcolai, the LinuxVTL worked fine until now. I keep as a last solution the deletion of the VTL and starting all over again with an empty one.

Thank you !

[Mark_Solutions]

If you cannot label or erase then there is something badly wrong with the system

Is there any form of message log on the VTL that you can look? or a set of configuration parameters to pass to it?

Starting again does sound to be favorite unless you can find what is causing the issue on the VTL

[Nicolai]

I am sorry to hear that labeling also fail.

Not having offcial vendor support has a down side. Maybe you can email the author of mhVTL and ask advice. Email can be found at https://sites.google.com/site/linuxvtl2/

[kronicker_chief]

Hello,

I have uninstalled / reinstalled Symantec Netbackup 7.5 on all the medias and master server, i have reverted on the linux VTL to an older VMware snapshot ( made not by me ), into a state in which the tapes are at 15 % written.

I have the same error.

I thought that the issue is caused by the fact that the tapes were written also by an old 2003 Media server that has been decomissioned, and replaced by a 2008, but reverting to this VTL snapshot brings me to the same error. This could still be the issue, meaning if i revert, i arriv into a state in which the tapes are written and handled aleardy but not by my environment.

I have sent an email to the vtl creator, maybe he can advise me how to completely erase & refresh these tapes so that NetBackup sees them brand new.

Thank you NetBackup virtuosos !

[kronicker_chief · Accepted solution]

Hello,

I connected a VTL from another environment and it worked, i can erase / label.

It seems that somehow i have logically damaged tapes / hearders in the past due to multiple tests and operations.

Thanks a lot for the help !