unable to start the NBU daemons

[loiver]

Hi,

We cannot start the NBU daemons on AIX 5.3 Media Server getting an errors:

UX01TEST:root[/usr/openv/netbackup/bin/goodies]./netbackup start
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/bpps because of th
e following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Database Server started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbevtmgr because o
f the following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Event Manager started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbemm because of t
he following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Enterprise Media Manager started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbrb because of th
e following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Resource Broker started.
exec(): 0509-036 Cannot load program /usr/openv/volmgr/bin/vmps because of the f
ollowing errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
Media Manager daemons started.
NetBackup request daemon started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/bpcompatd because
of the following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup compatibility daemon started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbjm because of th
e following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Job Manager started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbpem because of t
he following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Policy Execution Manager started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbstserv because o
f the following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Storage Service Manager started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbrmms because of
the following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Remote Monitoring Management System started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbsl because of th
e following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Service Layer started.
exec(): 0509-036 Cannot load program /usr/openv/netbackup/bin/nbsvcmon because o
f the following errors:
        0509-130 Symbol resolution failed for /usr/lib/librtl.a(shr.o) because:
        0509-136   Symbol skeytune (number 48) is not exported from
                   dependent module /unix.
        0509-136   Symbol _ukey_enable (number 94) is not exported from
                   dependent module /unix.
        0509-026 System error: Error 0
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
NetBackup Service Monitor started.
UX01TEST:root[/usr/openv/netbackup/bin/goodies]

Please help.

Thanks.

[CRZ · Accepted solution]

This LOOKS like a binary mismatch or possible corruption.  I'm assuming you have the correct AIX5 binaries installed, but I should never assume.  It worked at one time, right?  What version of NetBackup are we talking about here?  Did it stop working after an upgrade, or some other process which may have changed the binaries on the media server?

Does a quick check of the version files in /usr/openv/netbackup and /usr/openv/netbackup/bin give you a string containing the right Operating System?

Have you asked IBM to take a look at it?  If it's an ML issue we won't be as much help as they could be.