Accelerator Full Error "incorrect checksum"

[Stanleyj]

I have a single windows 2008 r2 server that is only used a file share.  It has about 300gb of standard user data nothing to fancy. Every so often i get the following error on my nightly full:

Job Details:

10/17/2012 8:53:04 PM - Info bpbkar(pid=1836) NOT using change journal data for <E:\>: unable to validate change journal usage <reason=previous backup wasn't a successful backup>
10/17/2012 9:02:48 PM - Critical bptm(pid=29919) A portion of data to be included from a previous backup (backupid = <ServerName>_1350430062, offset = 231717302272, length = 512) has incorrect checksum (calculated checksum from backup image 0731240601bd2e23ce6adc56654548cf9b65c6a1, expected checksum 9498495028971e749b16153c75eb3939f9ef2694)
10/17/2012 9:02:48 PM - Critical bptm(pid=29919) image write failed: error -1: plugin error     
10/17/2012 9:03:12 PM - Error bptm(pid=29919) cannot write image to disk, Invalid argument     
10/17/2012 9:03:12 PM - Info bptm(pid=29919) EXITING with status 84 <----------  

Windows Eventlog:

Faulting application name: bpbkar32.exe, version: 7.500.412.916, time stamp: 0x5055e1c4

Faulting module name: MSVCR100.dll, version: 10.0.40219.1, time stamp: 0x4d5f034a

Exception code: 0x40000015

Fault offset: 0x00000000000761c9

Faulting process id: 0x11c0

Faulting application start time: 0x01cdacc7f98a7614

Faulting application path: C:\Program Files\Veritas\NetBackup\bin\bpbkar32.exe

Faulting module path: C:\Windows\system32\MSVCR100.dll

Report Id: b7affe2c-18bc-11e2-addb-00219ba02d25

An incremental runs fine but when i do the full it fails at the same place with the same error.  To make the failure go away I manually run a full with rescan enabled but this may resolve it for 2 days or 2 weeks.  I never know.

I have about 100 different servers running acclerated backups and so far this is the only server that keeps having this error appear.

Is anyone else seeing something similar and possibly found a fis?  To me it looks like something on the windows side of it because i can reinstall the client it the error apears again at random.

Policy info:

Sun - Fri:

• 12pm = Incremental
• 6pm = Full (no rescan)

Sat:

• 12pm = Full (rescan enabled)

Change Journal is enabled

[Marianne]

Error seems to be on the media server:

Critical bptm(pid=29919) A portion of data to be included from a previous backup (backupid = <ServerName>_1350430062, offset = 231717302272, length = 512) has incorrect checksum ....

What kind of dedupe storage are you backing up to?
What is NBU patch level on media server?
 

[Stanleyj]

Hmm.

5200 appliance running 2.5.1 code.

I wonder why it only seems to happen for this one client/job.   

[Marianne]

We need Mark_Solutions here...

He had a similar problem not so long ago.
The issue was with a specific client that appeared in more than one policy with different case used for Client name in the policies. Problem only manifested after 2.5 upgrade.

[Stanleyj]

The client naming issue appeared for me also when i upgraded to 2.5.  Strangely enough when i applied the 2.5.1 patch some new clients appeared that i had to correct but this server was never one of those that expeirenced the issue.

This does look similar becuase its almost like it starts trying to pull info from the wrong image but running a rescan corrects it which validates the checksums again.  I dont know?  Thank you for helping.

[Marianne]

Hopefully Mark will be along soon to tell us how his problem was resolved...

[Mark_Solutions · Accepted solution]

Hi - I am here!

I didn't do the work myself as I am not a unix wiz but here is the basis of what was done ...

On the 5200 in the de-dupe area is a catlog section with each servers name and its metadata within that folder.

For a couple of clients (ones that appeared in more than one policy with their names in different case) they had two folders - matching the case as shown in the policies.

So all policies were edited so that the client name matched that shown on the Master server under netbackup\db\images\ (remember that this will cause all diffs to run as fulls) and then on the appliance a soft link was put in place between the two different case clients folders

I think all files were also moved to be in just the one folder then

This effectively meant that whichever folder the processes looked in it would find everything in both folders (hope that makes sense!)

I would not reccomend doing this yourself - i would ask Symantec Support to take a look and see if this is actualy the case and then do what ever is needed.

I didn't see if cause an app fault but the basic message is similar ... all backups worked but duplications /  restores would not work and this may be similar to what a synthetic / accelerator job is doing

Hope this helps

[Stanleyj]

Thanks Mark,

I had this problem when i first upgraded to 2.5 but this paticular server wasnt one of the ones i had issues with.  However I have created several policies using this server when i was first testing accelerator so it probably does have multiple directories created.  Thanks for the steps i will dig a little deeper and see if they can help.