Forum Discussion

ajojoy91's avatar
ajojoy91
Level 2
6 years ago

Cannot login to admin console using jnbSA

Hi,

Needed help with connection to the admin console.

Netbackup version 8.0

Master server: RHEL 6.2

I am attempting to open the admin console by directly connecting a display to the master server.

On attempting to open Netbackup admin console using /usr/open/java/jnbSA, I was getting the below error.

"NetBackup could not initiate a trust relationship with host hostname. Ensure that your Domain Name Service (DNS) resolves the hostname to the correct IP address and check network connectivity".

 

Found this article which stated it is due to nbatd service not running.

https://www.veritas.com/support/en_US/article.100014059

 

On checking the running services using ./bpps -x, I found that nbatd service is not found to be running. (Attachment- bpps output_1)

Tried restarting the services as mentioned in the article, 'If nbatd is not mentioned as running on the system, restarting NetBackup would solve the problem', it was not of any help as nbatd did not come up.

Tried a /usr/openv/netbackup/bin/nbatd as well, but no use.

Also, in the attachment (bpps output_1), we can see the nbwmc service running. However, on running bpps within the next 5 seconds, that service seems to be missing.

Then, tried the steps as mentioned in the marked solution in the discussion

https://vox.veritas.com/t5/NetBackup/Netbackup-could-not-initiate-a-trust-relationship-with-host/td-p/773792

Deleted the folder with the master server name, restarted services, and tried /usr/openv/netbackup/bin/jnbSA.

New error pops up

"The host does not have Netbackup Hostname based security certificate installed. The certificate is mandatory to establish a secure connection"

Reverted the changes from the backup copy of the files.

 

Tried the steps as mentioned in the marked solution of the discussion,

https://vox.veritas.com/t5/NetBackup/nbatd-is-failing-to-start/m-p/689744

 

Error:

[root@prdspnbapp01 bin]# /usr/openv/netbackup/bin/admincmd/bpnbaz -ConfigureAuth

You will have to restart NetBackup services on this machine after the command completes successfully.

Do you want to continue(y/n)y

Gathering configuration information.

ERROR: Failed to start security services - /usr/openv/netbackup/bin/nbatd -f

Unable to fork child process.

On searching some more, I stumbled upon this. (Attachment- nbcertcmd)

I found out that a certificate has expired on March 27, 2019. I tried steps to renew it, but to no avail.

I changed the system date to February, and the console started up, however with an error message stating that the Netbackup Web Management service is not running.

 

Could I have some help with getting nbatd and nbwmc up and running, and the admin console running without having to change the system date to some old date?

Regards,

Ajo

7 Replies

  • bpps output_1:

    [root@prdspnbapp01 bin]# ./bpps -a
    NB Processes
    ------------
    root 15585 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/vnetd -standalone
    root 15590 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/bpcd -standalone
    root 15607 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbdisco
    root 15725 1 42 11:50 ? 00:02:21 /usr/openv/db//bin/NB_dbsrv @/usr/openv/var/global/server.conf @/usr/openv/var/global/databases.conf -hn 5
    root 15796 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbevtmgr
    root 15803 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbaudit
    root 15927 1 0 11:50 ? 00:00:01 /usr/openv/netbackup/bin/nbemm
    root 15933 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbrb
    root 15964 1 0 11:50 pts/3 00:00:00 /usr/openv/netbackup/bin/bprd
    root 15983 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/bpcompatd
    root 15988 1 0 11:50 pts/3 00:00:00 /usr/openv/netbackup/bin/bpdbm
    root 15996 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbjm
    root 16001 15988 0 11:50 pts/3 00:00:00 /usr/openv/netbackup/bin/bpjobd
    root 16013 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbpem
    root 16034 16013 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem
    root 16044 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbstserv
    root 16083 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbrmms
    root 16168 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbsl
    root 16218 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbim
    root 16248 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbars
    root 16449 15988 0 11:50 pts/3 00:00:00 /usr/openv/netbackup/bin/bpdbm
    root 16450 15996 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbjm
    root 16526 1 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/nbsvcmon
    root 16797 16083 0 11:50 ? 00:00:00 /usr/openv/netbackup/bin/admincmd/bpstsinfo -DPSPROXY
    root 17138 16013 0 11:51 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem_cleanup
    nbwebsvc 18138 1 99 11:55 ? 00:02:40 /usr/openv/java/jre/bin/java -Dnop -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dnbwmc -Xrs -d64 -Xms512m -Xmx1024m -Djava.library.path=/usr/openv/lib -DNB_HOSTNAME=prdspnbapp01 -DNB_INSTALL_DIR=/usr/openv -DVAR_GLOBAL=/usr/openv/var/global -Djava.util.logging.config.file=/usr/openv/wmc/config/tomcat.logging.properties -Dvrts.NBJAVA_CONF=/usr/openv/wmc/config/nbj.conf -Dvrts.common.utilities.DEBUG_PROPERTIES=/usr/openv/wmc/config/Debug.properties -Dlog.conf.path=/usr/openv/netbackup/nblog.conf -Dvrtsat.home=/usr/openv/netbackup/sec/at -Dvrtsat.datadir=/usr/openv/var/vxss -Djdk.tls.ephemeralDHKeySize=2048 -Djdk.tls.rejectClientInitiatedRenegotiation=true -XX:ErrorFile=/usr/openv/wmc/webserver/logs/hs_err_pid%p.log -Djava.endorsed.dirs=/usr/openv/wmc/webserver/endorsed -classpath /usr/openv/wmc/webserver/bin/bootstrap.jar:/usr/openv/wmc/webserver/bin/tomcat-juli.jar -Dcatalina.base=/usr/openv/wmc/webserver -Dcatalina.home=/usr/openv/wmc/webserver -Djava.io.tmpdir=/usr/openv/wmc/webserver/temp org.apache.catalina.startup.Bootstrap start


    MM Processes
    ------------
    root 15944 1 0 11:50 pts/3 00:00:00 /usr/openv/volmgr/bin/ltid
    root 15951 1 0 11:50 pts/3 00:00:00 vmd -v
    root 16454 15944 0 11:50 pts/3 00:00:00 tldd -v
    root 16455 15951 0 11:50 pts/3 00:00:00 rdevmi -sockfd 14 -p 50 -r
    root 16471 15944 0 11:50 pts/3 00:00:00 avrd -v
    root 16481 1 0 11:50 pts/3 00:00:00 tldcd -v

     

     

    nbcertcmd:

    [root@prdspnbapp01 bin]# ./nbcertcmd -getCAcertificate
    nbcertcmd: The -getCACertificate operation failed for server prdspnbapp01.
    EXIT STATUS 8506: The certificate has expired.
    [root@prdspnbapp01 bin]# ./nbcertcmd -getCAcertificate
    nbcertcmd: The -getCACertificate operation failed for server prdspnbapp01.
    EXIT STATUS 8506: The certificate has expired.
    [root@prdspnbapp01 bin]# ./nbcertcmd -getCAcertificate
    nbcertcmd: The -getCACertificate operation failed for server prdspnbapp01.
    EXIT STATUS 26: client/server handshaking failed
    [root@prdspnbapp01 bin]# ./nbcertcmd -getcertificate
    EXIT STATUS 5950: Certificate already exists.
    [root@prdspnbapp01 bin]# ./nbcertcmd -listcacertdetails
    Subject Name : /CN=nbatd/OU=root@prdspnbapp01/O=vx
    Start Date : Jan 28 09:29:48 2014 GMT
    Expiry Date : Jan 23 10:44:48 2034 GMT
    SHA1 Fingerprint : AB:E5:C9:48:78:0C:2A:CF:69:2A:C7:9B:CC:93:A4:8A:39:06:4A:31

    Operation completed successfully.
    [root@prdspnbapp01 bin]# ./nbcertcmd -listcertdetails
    Master Server : prdspnbapp01
    Host ID : be819324-0697-43a6-9f92-957b74409288
    Issued By : /CN=broker/OU=root@prdspnbapp01/O=vx
    Serial Number : 0x6abadd960000000a
    Expiry Date : Mar 27 09:53:05 2019 GMT
    SHA1 Fingerprint : 58:AB:F0:07:F5:C4:59:9A:6C:F0:D0:24:5E:93:44:F5:58:F7:78:2F

    Operation completed successfully.

     

      • Amol_Nair's avatar
        Amol_Nair
        Level 6

        You really shouldn't be running bpnbaz configureAuth and other commands without really understanding what things it affects. With the new certificates in place, just running any such random command would eventually end up breaking everything.

        In your case the 1st thing that we need to concentrate on is getting nbatd running on the master server, without nbatd running the bpnbaz command failing is pretty understandable.

        Could you check if there are any files with .0 in the below path. You could directly run the below command to validate this information.

        ls -la /usr/openv/var/global/vxss/eab/data/root/.VRTSat/profile/certstore/*.0

        If you see a couple of such files kindly move them to say /tmp. *** Only the .0 files. Please do not move or touch anything else.

        Once you move it try to restart NetBackup services including pbx and check whether nbatd starts. If it still fails to start then I would say reviewing nbatd logs would give us the correct path to proceed ahead. 

    • Mike_Gavrilov's avatar
      Mike_Gavrilov
      Level 6

      Try to start nbatd manually:

      # export LD_LIBRARY_PATH=/usr/openv/netbackup/sec/at/lib/
      # /usr/openv/netbackup/bin/private/nbatd -c /usr/openv/var/global/vxss/eab/data

      and after that try to start WMC:

      # /usr/openv/netbackup/bin/nbwmc

       

      • ajojoy91's avatar
        ajojoy91
        Level 2

        Tried both:

        # export LD_LIBRARY_PATH=/usr/openv/netbackup/sec/at/lib/
        # /usr/openv/netbackup/bin/private/nbatd -c /usr/openv/var/global/vxss/eab/data

         Output after that:

        [root@prdspnbapp01 ~]# ps -ef|grep nbatd
        root 6254 14830 0 18:06 pts/1 00:00:00 grep nbatd
        [root@prdspnbapp01 ~]#
        [root@prdspnbapp01 ~]#
        [root@prdspnbapp01 ~]# /usr/openv/netbackup/bin/bpps -a
        NB Processes
        ------------
        root 1147 16013 0 Apr08 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem_email
        root 4613 16083 0 18:00 ? 00:00:00 /usr/openv/netbackup/bin/admincmd/bpstsinfo -DPSPROXY
        nbwebsvc 6347 1 99 18:06 ? 00:00:04 /usr/openv/java/jre/bin/java -Dnop -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dnbwmc -Xrs -d64 -Xms512m -Xmx1024m -Djava.library.path=/usr/openv/lib -DNB_HOSTNAME=prdspnbapp01 -DNB_INSTALL_DIR=/usr/openv -DVAR_GLOBAL=/usr/openv/var/global -Djava.util.logging.config.file=/usr/openv/wmc/config/tomcat.logging.properties -Dvrts.NBJAVA_CONF=/usr/openv/wmc/config/nbj.conf -Dvrts.common.utilities.DEBUG_PROPERTIES=/usr/openv/wmc/config/Debug.properties -Dlog.conf.path=/usr/openv/netbackup/nblog.conf -Dvrtsat.home=/usr/openv/netbackup/sec/at -Dvrtsat.datadir=/usr/openv/var/vxss -Djdk.tls.ephemeralDHKeySize=2048 -Djdk.tls.rejectClientInitiatedRenegotiation=true -XX:ErrorFile=/usr/openv/wmc/webserver/logs/hs_err_pid%p.log -Djava.endorsed.dirs=/usr/openv/wmc/webserver/endorsed -classpath /usr/openv/wmc/webserver/bin/bootstrap.jar:/usr/openv/wmc/webserver/bin/tomcat-juli.jar -Dcatalina.base=/usr/openv/wmc/webserver -Dcatalina.home=/usr/openv/wmc/webserver -Djava.io.tmpdir=/usr/openv/wmc/webserver/temp org.apache.catalina.startup.Bootstrap start
        root 15585 1 0 Apr08 ? 00:00:07 /usr/openv/netbackup/bin/vnetd -standalone
        root 15590 1 0 Apr08 ? 00:00:02 /usr/openv/netbackup/bin/bpcd -standalone
        root 15607 1 0 Apr08 ? 00:01:36 /usr/openv/netbackup/bin/nbdisco
        root 15725 1 9 Apr08 ? 09:49:04 /usr/openv/db//bin/NB_dbsrv @/usr/openv/var/global/server.conf @/usr/openv/var/global/databases.conf -hn 5
        root 15796 1 0 Apr08 ? 00:00:38 /usr/openv/netbackup/bin/nbevtmgr
        root 15803 1 0 Apr08 ? 00:01:13 /usr/openv/netbackup/bin/nbaudit
        root 15927 1 0 Apr08 ? 00:10:41 /usr/openv/netbackup/bin/nbemm
        root 15933 1 0 Apr08 ? 00:01:00 /usr/openv/netbackup/bin/nbrb
        root 15964 1 0 Apr08 pts/3 00:00:18 /usr/openv/netbackup/bin/bprd
        root 15983 1 0 Apr08 ? 00:00:03 /usr/openv/netbackup/bin/bpcompatd
        root 15988 1 0 Apr08 pts/3 00:00:03 /usr/openv/netbackup/bin/bpdbm
        root 15996 1 0 Apr08 ? 00:00:43 /usr/openv/netbackup/bin/nbjm
        root 16001 15988 0 Apr08 pts/3 00:00:31 /usr/openv/netbackup/bin/bpjobd
        root 16013 1 0 Apr08 ? 00:00:27 /usr/openv/netbackup/bin/nbpem
        root 16034 16013 0 Apr08 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem
        root 16044 1 0 Apr08 ? 00:01:04 /usr/openv/netbackup/bin/nbstserv
        root 16083 1 0 Apr08 ? 00:00:39 /usr/openv/netbackup/bin/nbrmms
        root 16168 1 0 Apr08 ? 00:01:01 /usr/openv/netbackup/bin/nbsl
        root 16218 1 0 Apr08 ? 00:00:26 /usr/openv/netbackup/bin/nbim
        root 16248 1 0 Apr08 ? 00:03:30 /usr/openv/netbackup/bin/nbars
        root 16449 15988 0 Apr08 pts/3 00:00:01 /usr/openv/netbackup/bin/bpdbm
        root 16450 15996 0 Apr08 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbjm
        root 16526 1 0 Apr08 ? 00:03:49 /usr/openv/netbackup/bin/nbsvcmon
        root 17138 16013 0 Apr08 ? 00:00:01 /usr/openv/netbackup/bin/nbproxy dblib nbpem_cleanup


        MM Processes
        ------------
        root 15944 1 0 Apr08 pts/3 00:00:12 /usr/openv/volmgr/bin/ltid
        root 15951 1 0 Apr08 pts/3 00:00:15 vmd -v
        root 16454 15944 0 Apr08 pts/3 00:00:00 tldd -v
        root 16471 15944 0 Apr08 pts/3 00:00:11 avrd -v
        root 16481 1 0 Apr08 pts/3 00:00:00 tldcd -v

         

        Nbatd is still not up.

        Nbwmc dies out after a while automatically.

        [root@prdspnbapp01 ~]# /usr/openv/netbackup/bin/bpps -a
        NB Processes
        ------------
        root 1147 16013 0 Apr08 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem_email
        root 4613 16083 0 18:00 ? 00:00:00 /usr/openv/netbackup/bin/admincmd/bpstsinfo -DPSPROXY
        root 15585 1 0 Apr08 ? 00:00:07 /usr/openv/netbackup/bin/vnetd -standalone
        root 15590 1 0 Apr08 ? 00:00:02 /usr/openv/netbackup/bin/bpcd -standalone
        root 15607 1 0 Apr08 ? 00:01:36 /usr/openv/netbackup/bin/nbdisco
        root 15725 1 9 Apr08 ? 09:49:05 /usr/openv/db//bin/NB_dbsrv @/usr/openv/var/global/server.conf @/usr/openv/var/global/databases.conf -hn 5
        root 15796 1 0 Apr08 ? 00:00:38 /usr/openv/netbackup/bin/nbevtmgr
        root 15803 1 0 Apr08 ? 00:01:13 /usr/openv/netbackup/bin/nbaudit
        root 15927 1 0 Apr08 ? 00:10:41 /usr/openv/netbackup/bin/nbemm
        root 15933 1 0 Apr08 ? 00:01:00 /usr/openv/netbackup/bin/nbrb
        root 15964 1 0 Apr08 pts/3 00:00:18 /usr/openv/netbackup/bin/bprd
        root 15983 1 0 Apr08 ? 00:00:03 /usr/openv/netbackup/bin/bpcompatd
        root 15988 1 0 Apr08 pts/3 00:00:03 /usr/openv/netbackup/bin/bpdbm
        root 15996 1 0 Apr08 ? 00:00:43 /usr/openv/netbackup/bin/nbjm
        root 16001 15988 0 Apr08 pts/3 00:00:31 /usr/openv/netbackup/bin/bpjobd
        root 16013 1 0 Apr08 ? 00:00:27 /usr/openv/netbackup/bin/nbpem
        root 16034 16013 0 Apr08 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbpem
        root 16044 1 0 Apr08 ? 00:01:04 /usr/openv/netbackup/bin/nbstserv
        root 16083 1 0 Apr08 ? 00:00:39 /usr/openv/netbackup/bin/nbrmms
        root 16168 1 0 Apr08 ? 00:01:01 /usr/openv/netbackup/bin/nbsl
        root 16218 1 0 Apr08 ? 00:00:26 /usr/openv/netbackup/bin/nbim
        root 16248 1 0 Apr08 ? 00:03:30 /usr/openv/netbackup/bin/nbars
        root 16449 15988 0 Apr08 pts/3 00:00:01 /usr/openv/netbackup/bin/bpdbm
        root 16450 15996 0 Apr08 ? 00:00:00 /usr/openv/netbackup/bin/nbproxy dblib nbjm
        root 16526 1 0 Apr08 ? 00:03:49 /usr/openv/netbackup/bin/nbsvcmon
        root 17138 16013 0 Apr08 ? 00:00:01 /usr/openv/netbackup/bin/nbproxy dblib nbpem_cleanup


        MM Processes
        ------------
        root 15944 1 0 Apr08 pts/3 00:00:12 /usr/openv/volmgr/bin/ltid
        root 15951 1 0 Apr08 pts/3 00:00:15 vmd -v
        root 16454 15944 0 Apr08 pts/3 00:00:00 tldd -v
        root 16471 15944 0 Apr08 pts/3 00:00:11 avrd -v
        root 16481 1 0 Apr08 pts/3 00:00:00 tldcd -v