Oracle backups doing very slow, potentionally failing with status 13

[frena]

Hello Netbackup experts!

I'm facing couple of weeks to problem with one client. There is running a oracle online backup to tape via sbt_tape

Oracle version is 10.2. running on solaris 10 client, netbackup version 6.5.6

Netbackup MS is 6.5.6 version, media server also, both arerunning on Solaris 10..

Problem is that backup is runnng very slow, but in strange way.

For example small control file with size 10MB...it is running in netbackup 3 minutes (wih mounting of tape it is perfectly fine)..job is finished successfully..

Problem seems to be on oracle site...there is job very slow on exit..and it takes 2,5 hour to finish..

I dont know for what it is waiting..According to oracle support it is waiting for netbackup...Try to guess what is a casue according to netbackup support ;)

If we are trying to backup oracle archive logs then we have problem with filling up archive log destination..Thing is that we are running archive jobs with a script from a client and every backup peace is starting twice...(in two schedules, intwo jobs)..it is managed by script..That means two jobs are starting paralell and one of them is finished in netbackup successully, but second one is many time running very long time and failed wtih status 13..  I think it is becasue of that script which is starting them is waiting till job  is closed also on oracle side and then close second job also on netbackup side...I'm not sure how is it working here but I asusme from test with controlfile backup that (above)  it is becasue oracle cannot close the backup job...

I found this technote: http://www.symantec.com/business/support/index?page=content&id=TECH130209

According this, we try to make backup of control file to sbt_disk and it rans perfectly well in few second..

However, control file with size 10 MB cannot be preallocated so I dont think that there is described real cause of problem

Filesystem backup is runnig there fine..CPU on the client is not overloaded, also disk and NIC seems to be ok..

We try to reinstall netbackup client software, netbackup agent...withouth help..

Symantec support recomends update to version 7.1 and make a clone of database to different client...and see how is it ding there..not possible due to customer rules...

Strange is that from netbackup side it looks perfectly well (expect of those archive jobs) and finishing with status 0 in time...But oracle has problem with close the backup...according the debug log it is waiting on netbackup...

In netbackup dbclient log is nothing to see for jobs which fisnihed in time...for archive logs there is somethine status 13..I cna attach logs if is somebody interesting..

Can somebody advice where should I look to or where can be a problem?

THX!

[Mick_Scott]

I'm a bit confused, but can you confirm the following

1. You aren't using the oracle agent?
2. from the client you can backup OS files to tape or disk storage unit ok?
3. when you say "archive jobs"  do you mean backup of oracle archive logs ?

Can you provide details of what the 2 scripts that you run in parallel are doing and why do they run in parallel ?

[Nicolai]

RMAN catalog fragmentation ?

If periodic recover catalog housekeeping hasn't been initiated some very basic operation may take a extended time, espically offline redo archives.

[Marianne]

Please show us the 'debug log' that you are mentioning?
We will also need the following NBU logs for the same backup attempt:
On client: dbclient
On media server: bptm and bpbrm
Please rename logs to reflect process name (e.g. dbclient.txt) and upload them as File attachments.

PS: You REALLY need to upgrade this environment - NBU 6.x ran out of support a week ago.

[frena]

Hi,

We are running backup via oracle agent..

Backup of oracle archive logs we are doing to tape storage unit...thing is it doesn't metter if it is to disk or tape storage unit..you can in rman chooce sbt_tape or sbt_disk...sbt_tape means that backup will be done via netbackup to storage unit which you have there, doesn't metter if it is disk, tape, advance disk or whatever....sbt_disk means that rman make backup wtihout netbackup to any disk which you have attached to the machine..

Scripts I can't provide right now..

About RMAN catalg fragmentation I ned to speak with our DB admin..but we are making online oracle archiving..

Logs are attached..

[frena]

And here is rman debug log..

[frena · Accepted solution]

Logs were deleted,

I created a index files according to http://www.symantec.com/business/support/index?page=content&id=HOWTO33396 and seems like is doing much better...