Netbackup error 41 + RMAN crosscheck

[Doctorski]

Hi,
We were experiencing occasional timeouts (error 41) on some of our backups for no obvious reason.

After some investigation I discovered that RMAN was flooding netbackup with requests for availibility of backup pieces during crosschecks, thus causing the timeouts. We have two databases at the moment using NBU + RMAN and are looking to roll this out further.

Our scripts do a crosscheck after each backup is successful. Is this too often. How often do others do the crosscheck ?

If you keep archive logs on a long retention this crosscheck problem will only grow.

HP-UX Master NBU 5.1 MP2

Any ideas or recommendations. Many thanks

[eric_lilleness]

Do:

#bperror -S 41 -r

look at suggestion #7

Are you sure your timeouts are set to correct values on Master/Media?Clent tiers? As i recall these should be set to 900 seconds

[Doctorski]

Eric,
Have been down that route many times. Timeout settings have been up to 3600 ! The problem really is rman causing the timeouts. Crosschecks were taking up to two hours to do. I moved the rman backups out of the way of normal backups, and hey presto. Everything ok.
As soon as I moved them back, occasional timeouts again.
I have expired a boat-load of no longer needed archive log backups and the crosschecks are now down to 15 minutes and everything is rosey. I had the luxury of being able to do this for the two db's in question.
However when we come to roll-out RMAN for some of our "core" db's, we will need to keep archive logs for up to 12 months. There will be more crosschecks and more archive logs.
I see the problem returning if there is no apparent answer.

Thanks anyway.

[Richard_Bannist]

wow, 2hour crosschecks is gonna hurt!. Not sure how much help i can be on this one (maybe not much as you have the 12month retention of archivelogs there); anyway, we only use crossheck on backups where we manually delete archivelogs outside of RMAN (on our test systems mainly); ie we never use crosscheck on backups that don't require manual deletion of archivelogs (by us outside of RMAN deleting them) (plus our archivelogs are only kept up to a month as any Oracle databases older than that are apparently of no use to the business).

Anyway, doubt if this will help but this is how we use crosscheck -

run {
allocate channel t1 type 'sbt_tape';
allocate channel t2 type 'sbt_tape';
allocate channel t3 type 'sbt_tape';

resync catalog;

change archivelog all crosscheck;

# Backup the Database, control file, archivelogs, then delete 1 copy of archive logs.
backup full database plus archivelog delete input;

release channel t1;
release channel t2;
release channel t3;
}

Rich

[Andreas_Almroth]

I think a common scenario is to split maintenance from backups. Crosschecks is in my opinion a maintenance activity.

Doing crosschecks every time you make a backup is not necessary, and if you backup your archive logs every hour it would not work obviously...

I usually schedule weekly crosschecks from the client side as a cronjob, which connects to NBU for maintenance. This is done during office hours when no/fewer backups are running. It is not taking much load of the system, it is just slow...

Still, client timeouts should be altered to a higher value, especially if you doing incremental backups in Oracle.

/A

[Doctorski]

Rescheduled crosscheck to a less busy period seems to have cured the timeouts.

Maybe we need a bigger boat !