Failed to load media management library. Additional Information : 2

[cs3472]

Hi All,

I am getting below error with RMAN.

Client : AIX

Netbackup : NetBackup-AIX53 7.1

Master server : SunOS 5.10 Generic_147440-01 sun4u sparc SUNW

Netbackup master server : NetBackup-Solaris10 7.1.0.3

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 10/20/2014 07:25:11
ORA-19554: error allocating device, device type: SBT_TAPE, device name:
ORA-27211: Failed to load Media Management Library
Additional information: 2

RMAN>

Recovery Manager complete.

Steps taken so far:

1. We have tried to relink the library.

2. Shutdown the DB and brought it online, relinked again.

3. Client machine was rebooted 80 days back.

Please let me know how to proceed further..

Note: The machine (client) acts as a media server as well.

[Marianne]

What is Oracle version?

Have you verified NBU/Oracle compatibility? 
NetBackup 7 Application/Database Agent (CL):  http://www.symantec.com/docs/TECH126904

Have you checked the output of oracle_link script on the client? (/tmp/make_trace.<pid> )
Please post this output file.

Have you verified that the link exists in $ORACLE_HOME/lib?
libobk.a must be linked to /usr/openv/netbackup/bin/libobk.a64 

[cs3472]

Hi Marianne,

Please find the required details below:

Mon Oct 20 07:17:54 CEST 2014
All Oracle instances should be shutdown before running this script.
Please log into the Unix system as the Oracle owner for running this script
Do you want to continue? (y/n) [n] y
LIBOBK path: /usr/openv/netbackup/bin
ORACLE_HOME: /products/oradb/rdbms1120_<dbname>/112020
Oracle version: 11.2.0.3.0
Linking LIBOBK:
Moving libobk.a to libobk.a.back
ln -s /usr/openv/netbackup/bin/libobk.a64 /products/oradb/rdbms1120_<dbname>/112020/lib/libobk.a
Done

Linked:

lrwxrwxrwx    1 usodbms  oinstall         35 Oct 20 05:58 libobk.a.back -> /usr/openv/netbackup/bin/libobk.a64
lrwxrwxrwx    1 usodbms  oinstall         35 Oct 20 07:17 libobk.a -> /usr/openv/netbackup/bin/libobk.a64

Oracle version used is 11g R2 which I see is supported in Netbackup 7.0 for the client machine.

[Marianne]

All looks fine from NBU point of view.

You need to find out why Oracle/rman is not finding the library.
Check that all variables are configured correctly in Oracle user profile, such as LD_LIBRARY_PATH and/or LIBPATH.

*** EDIT ***

I found something on an Oracle forum that indicates rman on AIX may be using for library name called libobk.a.1.
See https://community.oracle.com/thread/878497?start=0&tstart=0 
Try to create this link manually and see what happens.

I also found this NBU TN: http://www.symantec.com/docs/TECH194511 

So, check variables for Oracle user, or hard-code the SBT_LIBRARY location in the rman script:
 

Example of RMAN script:

RMAN> run {
ALLOCATE CHANNEL ch1 TYPE SBT CONNECT 'sys/syspassword@rac1' PARMS 'ENV=(NB_ORA_CLASS=test_policy),SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.a64(shr.o)' ;

}

[cs3472]

Thanks Marianne,

We have modified the script as informed :

connect target /
connect catalog rmanf/recocf@SS_RMANF
run
{
ALLOCATE CHANNEL ch1 TYPE SBT CONNECT 'usodbms/<userid>' PARMS 'ENV=(NB_ORA_CLASS=<policy name>),SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.a64(shr.o)' ;    
     backup
     full
     tag DB_DBS
     format 'db_s%s_p%p_t%t'
     (database);

}

But still we are getting the same error :

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of allocate command on ch1 channel at 10/20/2014 10:17:05
ORA-19554: error allocating device, device type: SBT_TAPE, device name:
ORA-27211: Failed to load Media Management Library
Additional information: 4005

[Marianne · Accepted solution]

I am out of ideas...

Maybe check if the NBU libobk is fine:

ls -l /usr/openv/netbackup/bin/libobk.a64 

and compare with output of similar client.

You may want to patch/upgrade the client to 7.1.0.3 or 7.1.0.4 as that should install a new version of the NBU library.

[Michael_G_Ander]

If this still is an issue, sbttest might help

Noticed that the link script asked to be run as the oracle owner

AIX being a 64 bit system , check if there is a lib64 folder rman might use in place of the lib folder

[Marianne]

Always frustrating when feedback in this forum simply stops....

I was wondering if the NBU libobk was fine. A corrupt binary will also have this effect.