first time - NBU install/DR exercise

[manatee]

i've got my test VM, obviously with lesser specs but i've managed to recreate the same partitions/folders that existed in my production NBU master/media server.

i'm installing 7.6 GA now and the screen has been like this for several minutes now (see below). shall i continue to wait or cancel?

The following automatic startup and shutdown scripts (respectively)
have been installed. They will cause the NetBackup daemons to be
automatically shut down and restarted each time the system boots.

/etc/rc.d/rc2.d/S77netbackup
/etc/rc.d/rc3.d/S77netbackup
/etc/rc.d/rc5.d/S77netbackup
/etc/rc.d/rc0.d/K01netbackup
/etc/rc.d/rc1.d/K01netbackup
/etc/rc.d/rc6.d/K01netbackup

Installation of Java LiveUpdate agent succeeded. Refer to file
/tmp/JLU-Log/JavaLiveUpdate-Install.log on test-nbu
for installation details.

Checking LiveUpdate registration for the following products: NB CLT
This may take a few minutes.

Product NB is installed and will be registered.
Product CLT_Linux-PSERIES is installed and will be registered.
Product CLT_Linux-ZSERIES is installed and will be registered.

Updating LiveUpdate registration now...this may take some time.

[manatee]

it's taking time so i cancelled and re-run the install script. this time it timed out after a few seconds. continuing, it spewed out the following messages. i can only guess that it's because i didn't start the installation in a separate network from the production NBU.

...

Machine test-nbu is a master server, and it is not the EMMSERVER.
Converting STREAMS files.  This may take a few minutes.

STREAMS files conversion is complete.


Successfully updated the session cache parameters.
..

Configuring nbatd.
Gathering configuration information.
Waiting for the security services to start operation.
Unable to setup trust with the Netbackup authentication broker. Please check that Netbackup is installed correctly.

Unable to contact Authentication server.
Creating /usr/openv/tmp/sqlany

Installed SQL Anywhere Version 12.0.1.3873
Installation completed successfully
..

Creating WSL server certificate ...
set_value: Key "AZDBPasswordFilePath" successfully updated
AZ database setup complete.

This is not a EMM and Master server, exiting

Migrating SLP definitions into the NetBackup relational database.

Starting the NetBackup Event Manager.


Starting the NetBackup Audit Manager.

Starting the NetBackup Deduplication Manager.
Starting the NetBackup Deduplication Engine.
Starting the NetBackup Deduplication Multi-Threaded Agent.

Starting the NetBackup database manager process (bpdbm).


ERROR:   Unable to start NetBackup daemon bpdbm.  Rerun
         /usr/openv/netbackup/bin/install_bp when the problem
         has been resolved.

[RiaanBadenhorst]

Did you set up hosts files in your test VM? Looks like a comms issue.

[manatee]

yes but only contains info for this server "test-nbu".

i'm separating its network now and will reinstall.

[Marianne]

Something wrong with hostname or hostname resolution of itself. 

Machine test-nbu is a master server, and it is not the EMMSERVER.
......
This is not a EMM and Master server, exiting

 What is output of 'hostname' for itself on the test master? 

Did you go through all the requirements in Installation Guide?
(You won't believe this, not so long ago we had someone here battling with NBU installation because Networking was not installed as part of OS installation!)

Do you have /etc/hosts entry for itself? 
27.0.0.1       localhost
<IP address>    <hostname>

What did you choose as hostname for EMM and Master during installation? 

[Marianne]

" .... yes but only contains info for this server "test-nbu"."

Master server hostname MUST match original master if you want to recover catalogs...

Same hostname, same installation path, same NBU & patch version...

[manatee]

hostname is "test-nbu" (not same sa NBU master name).

during install, it asked for master server name. i provided the same name as the current production master server.

don't recall it asking me for EMM name.

[manatee]

aha!

i already had that feeling but want to stretch the idea.

re-installing now.

[Marianne]

There is no point in carrying on if hostname does not match the original master name.

Master server daemons and processes will not start and you will not be able to recover the catalog.

The way you are installing is for a media server.

[manatee]

so recovery NBU server off the main network, reinstalled NBU, corrected the hostname issue. all went well. until..

Error: Unable to connect to Indexing Service.
EXIT STATUS = 1458

from what i can google, it's looking for some "NetBackup Indexing Service" so i presumed it's not yet up since it's a new install.

i rebooted the server.

[manatee]

no luck. even after a reboot.

/usr/openv/netbackup/bin/admincmd/nbindexutil -upgrade
Error: Unable to connect to Indexing Service.
EXIT STATUS = 1458

have to search what is this indexing service and how to bring it up.

[manatee]

got it! indexing service comes with media server and i didn't install media server coz i just want to bring up the NBU server and restore the catalog fast. like in a DR scenario.

seems media server is a required install. installing again.

[Marianne]

Seems I am missing something....

Did you manually run 'nbindexutil -upgrade' command? 

If so - why? 

As far as I can gather this command/service is related to NetBackup Search (Legal Search and Hold) option. 

[Marianne]

NetBackup Master server includes media server binaries and functionality.

NetBackup Search is a completely different option that is installed on a separate media server.

Do you have NetBackup Search in your environment? 

[manatee]

"NetBackup Search"? by that you mean as an additional license? i checked my existing licenses and no search doesn't show.

[manatee]

reinstalling didn't made any changes forward. got the same error and with a new one too:

<16>bpauthorize main: Function connect_to_remote_server(master) failed:48
client hostname could not be found

EMM interface initialization failed, status = 77
Starting the Media Manager device daemon processes.

/etc/hosts is fine with the hostname and ip address of the server. my thinking is it needs the interface up too. currently interface is down so not to conflict with existing network.

guess i'll have to connect the server back to the network but with a different IP or VLAN. (i was trying to do a DR restore using original settings).

[Marianne]

Again - I am trying to figure out what I am missing here...

Why are you concerned about indexing service if you don't have NetBackup Search?

Why are you running 'nbindexutil -upgrade' command? 

Have you compared processes/daemons running on this server with your production master (apart from Media manager processes)?

[manatee]

well the install is complaining about missing indexing service. should i move forward without fixing it?

no i didn't compare the daemons with the existing server as i'm trying to simulate a  total meltdown of production NBU server. like what i will only have are the installation files of NBU, and catalog. plus of course the tapes.

i'm following this link https://www.veritas.com/support/en_US/article.TECH65274 but trying to start on step 7. no GUI. just command line.

[manatee]

changed the ip address to class C. yes NBU likes the interfaces to be up and present during install.

after reboot, proceeded to do a

# /usr/openv/netbackup/bin/admincmd/bprecover -wizard

took 2 minutes to process the catalog DR file.

asks if i want to recover entire catalog. replied Y.

now it's asking should it start the Netbackup relational database (NBDB) after recovery. as this is database driven i answered Y.

catalog recovery in progress...

[manatee]

ERR - Failed to execute command /usr/openv/netbackup/bin/bpdm -restordir -jobid 2 -cn 1 -b master_1478149292 -c master (83)
Failed to recover catalog (83)

i'm now stumped. how does one troubleshoot this?

good thing this is only an exercise.