cancel
Showing results for 
Search instead for 
Did you mean: 

RMAN Hot Database backup fails with multiple tape channel allocations

kfh
Level 3

I have a client running an incremental hot database backup which used to work when the backup ran in a NetBackup 5.1 enterprise but doesn't in my new NetBackup 6.5.4 enterprise.  The rman script allocates 3 tape channels.  Tapes in a VTL are mounted and the master logs indicate the tapes are open for writing.  However only the first channel writes while the other 2 fail with the following errors:

RMAN-03009: failure of backup command on t2 channel at 12/02/2009 06:57:25
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
   Failed to process backup file <......>

The message for t3 is the same. Both channels t2 and t3 are disabled and then channel t1 backs up the failed t2 and t3 files, and the backup ultimately succeeds using channel t1 for the entire backup.

Here is part of the Daily Incremental section of the script:
if [[ $SCHED = "Daily" ]]
then
   /usr/local/bin/markup mydatabase
   echo "Starting RMAN script consistent level 1" `date`
   $ORACLE_HOME/bin/rman <<RMAN_EOF

   connect target
   connect catalog ${CATALOG_CONN}

   run {
        #hot database level 1 consistent backup
        allocate channel t1 type 'SBT_TAPE'
                 parms="ENV=(NB_ORA_SCHED=Daily-Policy)";
        allocate channel t2 type 'SBT_TAPE'
                 parms="ENV=(NB_ORA_SCHED=Daily-Policy)";
        allocate channel t3 type 'SBT_TAPE'
                 parms="ENV=(NB_ORA_SCHED=Daily-Policy)";
backup
           incremental level 1
           cumulative
           tag = 'Daily_Bkup_Level1'
           filesperset 3
           format '%d_DAILY_%U'
                (database);

        backup current controlfile
           format '%d_CTRL_%U'
           tag = 'Ctrl_Bkup' ;

        backup (SPFILE
           format = '%d_SPFILE_%U'
           tag = 'Spfile_Bkup') ;

Any help is appreciated.


7 REPLIES 7

Mouse
Moderator
Moderator
Partner    VIP    Accredited Certified

how many jobs per client are permitted in the master's settings?

kfh
Level 3
99 jobs per client.

The RMAN script allocates 3 SBT_TAPE channels and with each allocation handling 3 filesperset.

What's weird is that the allocation of the 3 channels works just fine for cold database backups - just not the hot backups (which are incrementals)

Rakesh_Khandelw
Level 6
Did you upgrade client to 6.5.4 as well and if you have already done that make sure NetBackup agent for oracle on client is upgraded to 6.5.4 as well.

kfh
Level 3
client and agent are at 6.5.4.

hbrits
Level 2
Partner Accredited Certified
I had simmelar problems where the 3rd and 4th stream failed and then was disabled. We solved it by changing /etc/xinetd.conf. Have a look at this link : http://www.freelists.org/post/oracle-l/SOLUTION-status-25-errors-with-Netbackup-Oracle-on-Linux,1

There is an entry in that file :  “per_source   = 10”

Change the “10” to “UNLIMITED“ or anything greater than 10.

Restart the service:

service xinetd restart



Henri

Symontec
Level 5
I wanted to know if you finally solved this issue or if there is something we can do to help you.

Regards,
SAF

kfh
Level 3
It turns out that there was a hardware firewall in front of the client which NAT'ed the public IP address to a private address.  I was able to get an IP address on the private network for the backup server so that backup traffic was no longer going through the firewall.  The multi-stream level 1 backup now works but is ridiculously slow.  When I compared the speeds of this backup with what it was when the client was being backed up in a NB5.1 enterprise, I found that the backup speeds were the same.

Level 1 incrementals are just slow, but at least, now the hardware firewall isn't killing multiple-channel sockets.

During the course of troubleshooting the multiple-channel level-1 backup failures, Symantec had the client enable Block Change Tracking (BCT).  The BCT
backups were fast and small, and the DBA will be implementing it on all of his Oracle backups as soon as he tests restorability of the BCT backups; right now his
workload won't allow it.