SAP agent -- RMAN failures

[Nicholas_Wells]

Friends,

We have just recently installed Netbackup 6.0 MP5 Enterprise server.  Also netbackup agent for SAP and Oracle on our client system.  I followed the netbackup documentation to configure the SAP agent to work with the master server and the tape library.  I created/edited the following files:

/oracle/PX2/102_64/dbs/initPX2.utl

/oracle/PX2/102_64/dbs/initPX2.sap

/usr/openv/netbackup/ext/db_ext/sap/scripts/sap_oracle/sap_rman_backup.new

If I completely bypass netbackup and run my brtools straight to my disk using rman parameters, I have no errors.

When I kick off the script sap_rman_backup.new, I get the following error from RMAN:

RMAN> 2> 3> 4> 5>
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of allocate command on sbt_1 channel at 09/28/2007 20:38:57
ORA-19554: error allocating device, device type: SBT_TAPE, device name:
ORA-27211: Failed to load Media Management Library
Additional information: 25

I have no idea what I can do about this error.  I know that we have LTO drives in our IBM tape storage system.

Here are the relevant file entries in my config file(s):

/oracle/PX2/102_64/dbs/initPX2.utl

switch_list /oracle/PX2/sapbackup/.switch.lis

switch_sem /oracle/PX2/sapbackup/.switch.sem

switch_log /oracle/PX2/sapbackup/.switch.log

backint_dir /var/tmp/PX2/sapscript/backint_dir

server redsa0011p00314

client plmt20000101

policy SAP
schedule Sap-Weekly

/oracle/PX2/102_64/dbs/initPX2.sap

backup_dev_type = util_file
rman_parms = "ENV=(NB_ORA_SAP=/oracle/PX2/102_64/dbs/initPX2.utl)"
util_par_file = ?/dbs/init@.utl

Also, here in the script I am running:

/usr/openv/netbackup/ext/db_ext/sap/scripts/sap_oracle/sap_rman_backup.new

#!/bin/sh

#bcpyrght
#***************************************************************************
#* $VRTScprght: Copyright 1993 - 2007 Symantec Corporation, All Rights Reserved $ *
#***************************************************************************
#ecpyrght

#
#NOTE:IF your SAP user (in this script sapora) runs in C shell, environmental
#variables can not be exported. In that case, you should modify this script to
#work in your environment. For example:
#   SAP_SERVER=$SAP_SERVER; export SAP_SERVER; (Correct for Bourne and Korn shells)
#   can change into
#   setenv SAP_SERVER $SAP_SERVER; (Correct for C shell)
#

#
#This environment variable are created by Netbackup (bphdb)
#

echo "SAP_SCHEDULED = $SAP_SCHEDULED"
echo "SAP_USER_INITIATED = $SAP_USER_INITIATED"
echo "SAP_SERVER = $SAP_SERVER"
echo "SAP_POLICY = $SAP_POLICY"
echo "SAP_FULL = $SAP_FULL"
echo "SAP_CINC = $SAP_CINC"
echo "SAP_SCHED = $SAP_SCHED"

RETURN_STATUS=0

SET_ENV=""

#
# If SAP_SERVER exists then export it to make it available to backint
#
if [ -n "$SAP_SERVER" ]
then
#        SAP_ENV="$SAP_ENV SAP_SERVER=$SAP_SERVER; export SAP_SERVER;"
#If Oracel DBA account( orapx2  user) uses C Shell, comment the above line and uncomment the next line
       SAP_ENV="$SAP_ENV setenv SAP_SERVER $SAP_SERVER;"

fi

#
# If SAP_POLICY exists then export it to make it available to backint
#
if [ -n "$SAP_POLICY" ]
then
#        SAP_ENV="$SAP_ENV SAP_POLICY=$SAP_POLICY; export SAP_POLICY;"
#If Oracel DBA account( orapx2  user) uses C Shell, comment the above line and uncomment the next line
       SAP_ENV="$SAP_ENV setenv SAP_POLICY $SAP_POLICY;"
fi

#
# If SAP_SCHED exists then export it to make it available to backint.
#
if [ -n "$SAP_SCHED" ]
then
#        SAP_ENV="$SAP_ENV SAP_SCHED=$SAP_SCHED; export SAP_SCHED;"
#if Oracle DBA account(orapx2 user) uses C Shell, comment the above line and uncomment the next line
       SAP_ENV="$SAP_ENV setenv SAP_SCHED $SAP_SCHED;"
fi

if [ $SAP_FULL -eq 1 ]; then
        CMD_LINE="$SAP_ENV brbackup -p /oracle/PX2/102_64/dbs/initPX2.sap -u / -d rman_util -t online_cons -m full -c"
elif [ $SAP_CINC -eq 1 ]; then
        CMD_LINE="$SAP_ENV brbackup -p /oracle/PX2/102_64/dbs/initPX2.sap -u / -d rman_util -t online_cons -m incr -c"
fi

#
# The username on the "su" command needs to be replaced with the correct
# user name.
#
echo "Execute $CMD_LINE"
su - orapx2 -c "$CMD_LINE"

RETURN_STATUS=$?

exit $RETURN_STATUS

Any Help would be GREATLY APPRECIATED. 

Thank you,

Nick

[Jim-m]

Hi

NOT a NBU issues - This is a fix for the DBAs -  SAP: Oracle: RAMAN appears to be at fault.

The "library" refered to was probably the runtime excutable code library, not a tape library.

I think a  SAP installation of oracle should be no different than the native oracle. See online Oracle RMAN manual :http://download-uk.oracle.com/docs/cd/B19306_01/backup.102/b14191/rcmtroub001.htm#sthref1134

The key part of the RMAN log are:.

ORA-nnnn (Where n is a digit) These are Oracle messages.  The media manager is the oracle media manager component of RMAN, not third party media manager i.e not a part of netbackup.

ORA-19511 if found indicates the problem source is the third party software (Netbackup).  Then followed by  the

third party software (Netbackup) messages returned to RMAN. 

They will look like this - the netbackup errors should be familiar to any NBU admin

ORA-19511: Error received from media manager layer, error text:

VxBSACreateObject: Failed with error:

   Server Status: unable to allocate new media for backup, storage unit

   has none available

-----------------------------------------------------------------------------------------------------

ORA-19511: Error received from media manager layer, error text:

VxBSACreateObject: Failed with error:

   Server Status:  Communication with the server has not been initiated

   or the server status has not been retrieved from the server.

------------------------------------------------------------------------------------------------------

ORA-19511: Error received from media manager layer, error text:

VxBSACreateObject: Failed with error:

   Server Status: the specified client does not exist in the specified

   policy

etc

FYI : sometime NGU gets confused and labels some rman backups as being successful even though they have failed internally with rman.    Looks like the return code doesn't reach NBU cause the return code of the NBU operation probably implies that it was successful

Message Edited by Jim-m on 09-30-2007 03:37 PM

[Nicolai]

Hi Nicholas.


You need to set backup_dev_type  to rman_util in order to use rman. If backup_dev_type = util_file SAP talk to Netbackup via backint.

Best Regrads

Nicolai

[Nicholas_Wells]

Jim-m,

Thanks very much for your posting.  I will pass this info along to my DBA and we'll take a closer look at it.

Nicolai,

I tried changing the init{SID}.sap to the backup_dev_type  to rman_util parameter.  I still got the same error.

I have one additional question.  The netbackup sap agent documentation says to create a symbolic link like so:

ln -s /usr/openv/netbackup/bin/libsapora.so  $ORACLE_HOME/lib/libobk.so

But in our version of netbackup (6.0 MP5), we don't have the libsapora.so  file.  We have these:

libsapora.so.1
libsapora.so64.1

We used the libsapora.so.1 and not the "64".  We weren't sure what to do here.  is that OK?

Thanks,

Nick

[Nicolai]

Hi agin.

Since you're Oracle path is  /oracle/PX2/102_64/dbs/  I would say you need to use the libsapora.so64.1 file.

There are also a couple of other tings to check:

Permissions of libsapora.?? (both file and directory)
Is /usr/openv/netbackup/logs/user_ops  & dbclient writable for the dba group
Does it work if you use the generic Oracle module (libobk.??)  - test with this small script (as the oraSID user).
 
rman nocatalog
  

run {

allocate channel ch00 type 'sbt_tape';

send 'NB_ORA_CLIENT=<source client>;

backup current controlfile;

release channel ch00;

}

Oracle/SAP is picky about the placement of  the symbolic link.

In 6.5 policy type needs to be SAP for backint and brbackup/rman backups. If RMAN is called direct, policy type need to be ORACLE. Failure to do so result in error code 239.

[Nicholas_Wells]

Nicolai,

Once again, thank you for the information.  We'll try changing the sym link to "64" and checking the permissions on the folder.  I'll let you  know.

Nick