BE v9.1 increasing failures differences in files during verify

[Paul_Iossi]

Hi,

I have had 19 consecutive backup failures now with the cause being 0x0 other errors - specifically during the verify phase it reports differences in file sizes on a few files. The files vary day to day and are not the kinds of files that are subject to being used and changed during the backup in progress (which is during the wee hours, no users), they are archived data not in use, etc. I have tried cleaning cycles, a fresh tape, nothing seems to get me back to a successful backup. I used to get this from time to time but now it seems to be permanent. Any ideas on causes and how to resolve?

Thanks,

Paul.

[Ken_Putnam]

To start with Backup Exec does not compare tape to disk. Verify merely reads the tape, recalculates checksums and compares them to the checksums written to tape

Do you have any mfg diagnostics that you can run against the drive?
Have you checked cables and terminators?
Do you have another SCSI controller you can try?

[Paul_Iossi]

I have run Exabyte's VXA-2 diagnostic tools on the drive which report no problems. Cables are optimal, LVD grade, properly terminated, etc.. The SCSI controller is not the likely candidate either, nothing has changed in terms of hardware configuration between the time that the backups were successful and the current crop of failed backups. The tape drive is the only device on the controller and the only SCSI device on the server - disks are SATA. Cleaning the heads and use of a fresh new tape did not resolve the issue. There have been no software installations or patches during that interval either and no configuration changes within Backup Exec. The same recurring scheduled backup job just started reporting file differences on a few files during the verify phase and reporting the backups as failed. The tape drive is just over a year old. The tapes being used are pretty new and nowhere near their 500 use limit specification.

[Amruta_Purandar]

Hello,

Please specify the errors that you are facing. You may meanwhile refer the following technote and check your configuration.

Title:
How to troubleshoot issues with a Robotic Library or Tape Drive in Backup Exec 9.x and 10.0
http://support.veritas.com/docs/255501

NOTE : If we do not receive your reply within two business days, this post would be marked assumed answered and would be moved to answered questions pool.

[Paul_Iossi]

I had previously specified the errors I am facing but will happily repeat them. Tape backups over the last 19 days have reported failures. In reviewing the logs, each time the backup completes but during the verify phase Backup Exec reports finding a few files with differences leading to failures - error code 0x0 other errors. These files vary from job to job. The files are static files rather than dynamic files that would be in use (so they are not actually changing during the course of backup operations). All hardware and software configurations are unchanged during the period of time during which the backups transitioned from successful to unsuccessful. Cleanings and the use of a brand new tape unsuccessfully would seem to preclude worn media, etc.

[Amruta_Purandar]

Hello,

- Are there any events logged in the event log? For example event ids 7, 9, 11 and/or 15.
- Are there any alerts?
- Other than the error "0x0 other errors" is there a more specific error message?


NOTE : If we do not receive your reply within two business days, this post would be marked assumed answered and would be moved to answered questions pool.

[Richard_Jones_6]

I'm also having this issue that popped up out of the blue and with a quick google search it seems a growing number of people are having the same problem. I noticed this issue had been patched in the past and was thinking that some old code had made it back into one of the new patches.

[Paul_Iossi]

Here is the alert that results following failure:

Backup 0001 -- The job failed.
Category: Job Failed
Type Error
Server: SUMMIT-DC
Device: EXABYTE 1
Job name: Backup 0001
Time alert received: 7/1/2005 3:26:18 AM
Source: Job

Properties of Category: Job FailedEnabled: Yes
Send notifications: Yes
Record in event log: Yes
Event ID: 34113
Automatically clear: No
Include job log: No

This is from the job log (completion status section):

Job ended: Friday, July 01, 2005 at 3:26:18 AM
Completed status: Failed
Final error: 0x0 -
Final error category: Other Errors

This is from the job log (errors section):

Click an error below to locate it in the job log
Verify - C: Difference encountered in file data.
The file \Company Shared Folder\Archived files, data, misc\Misc Odds & Ends\stuff\O2003 install\FILES\OWC10\SETUP.EXE is different.
Difference encountered in file data.
The file \Company Shared Folder\Archived files, data, misc\Misc Odds & Ends\stuff\Trial Backup Exec\Q118478.BEWS.91.4691.1_264658\WINNT\INSTALL\SAPR3Inst\VERITAS Backup Exec Agent for R3 for Oracle Server.msi is different.
Difference encountered in file data.
The file \WINDOWS\Installer\c299177d.msi is different.

As you can see the files are not the type that would be subject to changing in mid-backup. Additoinally the files vary day to day.

I will check the event viewer momentarily and report back any events logged in reference to your question.

Thanks,

-Paul.

[Barrey_Jewall]

I'm seeing nearly identical errors on two clients of mine since we did an audit and turned on verify.

One is running 9.1, the other is running 8.6.

Both have the latest patches publically available.

One client is using an HP DLT VS80, the other an HP Ultrium.

Both drives are connected via an Adaptec 29160LP.

At HP's suggestion, I changed the data rate on the controller hosting the VS80 to 40MB, from 160. (the drive autonegotiates to 20 sync (40) anyway, so this would not impact performance.

All cables are high quality, and have been checked, all termination is in place. These drives are the only devices on their SCSI controllers.

Both Drives pass whatever diags myself and HP can think of throwing at them, no problems.

This is REALLY getting frustrating for us, and expensive for our clients.

-Barrey

[Paul_Iossi]

I have checked the event log. No 7, 9, 11 or 15 errors reported. I did find a reference to a different error message that cropped up a couple of days ago when a job hung at a certain point: "The job failed with the following error: A failure occured querying the writer status". I had to cancel that particular job because it wouldn't progress past a certain point, approximately 675,000 bytes or so. This is unrelated to the usual failures however. This was an isolated different kind of failure.

[Barrey_Jewall]

UPDATE:

At one of our clients, we replaced the tape drive, no change. We removed BE 9.1 and installed the latest BE10, no change. Using NTBackup, no errors are reported, with verify turned on. The only client I can think of with BE verify turned on and having no errors is a client running an old patchlevel of BE 9.0.

Also, a previous (fixed) bug in BE 9.0 has caused this exact error message to appear under certain circumstances.

[Melvin_Ho]

Hi guys,

My clients has the same error too for the past few days. Also v9.1 All the backup job ended as Failed but a check in the restoration window, it seems that everything is backed up.
Is it a tape drive problem or a software problem? How can I resolve this?

Thanks.

Melvin

[Renuka_-]

Hello Paul,

Backup exec does no compare the backed up files to those on the disk.PLease refer to the following technote for further clarification:
http://support.veritas.com/docs/192163
If you are receiving errors during verfification it should be a problem with the device.
Please ensure that the tape is not using a RAID controller.
Ensure that you have the latest drivers and firmware.
check the scsi cables and scsi ermination.
try backups to disk to isolate the problem.

You have mentioned that "
I have checked the event log. No 7, 9, 11 or 15 errors reported." does this mean you have any of these event ids or do you not?


Additional Information :
For information on the recent VERITAS Backup Exec security vulnerabilities, including links to the downloads for the necessary hotfixes, please refer to the following document:
Patch summary for Security Advisories VX05-001, VX05-002, VX05-003, VX05-005, VX05-006, VX05-007

http://seer.support.veritas.com/docs/277429.htm

NOTE : If we do not receive your reply within two business days, this post would be marked assumed answeredand would be moved toanswered questions pool.

[Barrey_Jewall]

I'm not Paul, but here is the way it is going at one client:

NO (that is, Not ANY) of the listed errors in the event log.
NOT connected to a RAID controller (Adaptec 29160LP).
Adaptec is flashed to ver 3.10 (latest flash)
Using Adaptec SCSI drivers SP5-S4 released 6/2005.
(have tried Microsoft and older Adaptec drivers as well)
Using current veritas driver for tape drive (External HP VS80).
Have checked and replaced SCSI cables and terminator.
HP has replaced tape drive, even though it did NOT fail diags.
We don't currently have enough diskspace there to backup to disk.
Tests using NTbackup have not failed, with verify turned on.
Oh yeah, we completely uninstalled 9.1 and installed a new, downloaded copy of 10 in eval mode.

None of this has made any difference, client is still getting errors during the verify.

-Barrey

[Renuka_-]

Hello Barrey,

After installing the V10.0 have you recreated the .dr files? The idr preparation files wre they created using v9.1.


Additional Information :
For information on the recent VERITAS Backup Exec security vulnerabilities, including links to the downloads for the necessary hotfixes, please refer to the following document:
Patch summary for Security Advisories VX05-001, VX05-002, VX05-003, VX05-005, VX05-006, VX05-007

http://seer.support.veritas.com/docs/277429.htm

NOTE : If we do not receive your reply within two business days, this post would be marked assumed answeredand would be moved toanswered questions pool.

[Barrey_Jewall]

Not sure what effect this would have, but no, we did not re-create the .dr files. They MAY have been created by the new install, although I seem to recall that they aren' created until you run the IDR wizard??

What effect would this have unless we were doing an IDR backup, anyway?

However, what solved this problem at one site was to replace the Adaptec 29160ALP with a 2940U2W.

No, I don't know why it worked.

- Barrey

[Deepali_Badave]

Hello,

As you have mentioned here that the problem has resolved at one site, can u elaborate the steps you have followed?

NOTE : If we do not receive your reply within two business days, this post would be marked assumed answered and would be moved to answered questions pool.

[Barrey_Jewall]

What part of "what solved this problem at one site was to replace the Adaptec 29160ALP with a 2940U2W" requires clarification?

Please advise what you don't understand and I will try to explain it.

-Barrey