Forum Discussion

Krutons's avatar
Krutons
Moderator
5 years ago

8.2 Client Upgrade

I recently upgraded an environment from 8.1.1 to 8.2 and had no issues except with 3 linux clients. When I tried to push the client upgrade using the deployment policies, I get a status 7226. I also tried to manually install the client and got the same error.

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Finished populating answer file.

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Reading NetBackup version from version file...

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Detected version:

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) 8.1.1

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) 

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Executing [rpm -U /usr/openv/tmp/nbclient_8.2_redhat_x64.sja.extracted/VRTSnbpck.rpm 2>&1]...

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Execution finished.

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) =======Output from command=======

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378)         package VRTSnbpck-8.2.0.0-1561464247.x86_64 is already installed

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) =================================

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Installation finished with status [1].

Feb 7, 2020 11:32:49 AM - Info RUNCMD (pid=10378) Executing [rpm -U /usr/openv/tmp/nbclient_8.2_redhat_x64.sja.extracted/VRTSpbx.rpm 2>&1]...

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) Execution finished.

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) =======Output from command=======

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) 

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) ERROR:   The VRTSnbpck package did not install successfully.

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378)          Please look at the /tmp/install_VRTSnbpck_trace.30471

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378)          install trace for more details.

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) error: %pre(VRTSpbx-1.11.17.1-1.x86_64) scriptlet failed, exit status 1

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) error: VRTSpbx-1.11.17.1-1.x86_64: install failed

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) error: VRTSpbx-1.8.7.0-0.x86_64: erase skipped

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) =================================

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) Installation finished with status [1].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) agent_set_inherit_flag: Successfully set inherit flag 0 on 0[close on exec first 256].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) agent_set_inherit_flag: Successfully set inherit flag 0 on 1[close on exec first 256].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) agent_set_inherit_flag: Successfully set inherit flag 0 on 2[close on exec first 256].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) agent_set_inherit_flag: Successfully set inherit flag 0 on 7[close on exec first 256].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) agent_set_inherit_flag: Successfully set inherit flag 0 on 8[close on exec first 256].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) agent_set_inherit_flag: Successfully set inherit flag 0 on 13[close on exec first 256].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) Attempt to clear VNET_DOMAIN environment variable returned status [0].

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) Running startup command: [/usr/openv/netbackup/bin/bp.start_all 2>&1].

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Execution finished.

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Command returned following output:

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) ------------------------------------------------------------

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Starting vnetd...

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Starting bpcd...

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Starting nbftclnt...

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Starting nbdisco...

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Starting bmrbd...

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) 

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) ------------------------------------------------------------

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Successfully restarted NetBackup daemons.

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Removed [/usr/openv/netbackup/bin/nbliveup].

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Restored filename [/usr/openv/netbackup/bin/nbliveup.vxupdate.renamed] to [/usr/openv/netbackup/bin/nbliveup].

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) EXIT STATUS 7226

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Performing terminating close of sock(2)

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Performing terminating close of sock(1)

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) Execution of nbinstallagent on the remote host failed

Feb 7, 2020 11:32:52 AM - Info RUNCMD (pid=10378) exiting with status: 7226

Operation Status: 7226

 

I see this message but before that, it says that it's already installed (from previous attempt I take it).

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) ERROR:   The VRTSnbpck package did not install successfully.

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378)          Please look at the /tmp/install_VRTSnbpck_trace.30471

But the trace file says it was installed successfully (because it was installed on a previous attempt) and the VRTSpbx trace says there was a failure.
Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) error: %pre(VRTSpbx-1.11.17.1-1.x86_64) scriptlet failed, exit status 1

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) error: VRTSpbx-1.11.17.1-1.x86_64: install failed

Feb 7, 2020 11:32:50 AM - Info RUNCMD (pid=10378) error: VRTSpbx-1.8.7.0-0.x86_64: erase skipped

[root@<client name> netbackup]# rpm -qa |grep VRTS
VRTSpbx-1.8.7.0-0.x86_64
VRTSnbpck-8.2.0.0-1561464247.x86_64
VRTSnbjava-8.1.1.0-1517731855.x86_64
VRTSnbjre-8.1.1.0-1517731822.x86_64
VRTSnbclt-8.1.1.0-1517731893.x86_64
VRTSnbcfg-8.1.1.0-1517731818.x86_64
VRTSpddea-11.1.1.0-0061.x86_64

 

 

When I try to manually run the install, I see the following.

 

Do you wish to continue? [y,n] (y) y

The private_tmp value is /tmp/nbcheck_dir.29466

Checking for required system conditions...

/netbackup/NetBackup_8.2_CLIENTS2/NBClients/anb/Clients/usr/openv/netbackup/client/Linux/RedHat2.6.32/nbcheck returned exit status 255.

WARNING: Unable to provide preinstall check results.

Do you want to install the NetBackup client software for this client? [y,n] (y) y

This package will install Linux/RedHat2.6.32 client.

NetBackup client is currently at release level 8.1.1.

This package will install NetBackup client 8.2.

Successfully unpacked /netbackup/NetBackup_8.2_CLIENTS2/NBClients/anb/Clients/usr/openv/netbackup/client/Linux/RedHat2.6.32/certcmdTool_for_UNIX.tar.gz.

Getting CA certificate mode from master server.

NOTE: Depending on the network, this action may take a few minutes.

To continue without setting up secure communication, press Ctrl+C.

Master server <master server> security configuration is unknown.

Checking for local NBCA certificate

Checking for local ECA certificate

Local host certificate status is indeterminate.

WARNING: Could not determine the certificate status of either

the master server or the local host.

Do you wish to proceed with installation even though NetBackup may not operate? [y,n] (n) y

WARNING: Proceeding with the installation. Manual steps may be required before backups

and restores can occur. For more information:

For external CA, refer: https://www.veritas.com/support/en_US/article.100044300

For NetBackup CA, refer: https://www.veritas.com/support/en_US/article.000127129

Client binaries are located in /netbackup/NetBackup_8.2_CLIENTS2/NBClients/anb/Clients/usr/openv/netbackup/client/Linux/RedHat2.6.32.

Starting NetBackup Deduplication installer

testing: /tmp/pdde_pkg_dir_29669

Checking for SYMCpddea package...

Checking for VRTSpddea package...

Package VRTSpddea found.

NetBackup Deduplication software is installed.

NetBackup Deduplication is not yet configured.

NetBackup Deduplication preinstall check passed

Terminating NetBackup and/or Media Manager processes.

Installing VRTSnbpck ...

Installation of VRTSnbpck was successful. Check /tmp/install_VRTSnbpck.rpm_trace.29651 for details.

Installing VRTSpbx ...

Failed to install VRTSpbx status = 1.

Check /tmp/install_VRTSpbx.rpm_trace.29651 for details.

Aborting ...

ERROR: A failure was detected running

/netbackup/NetBackup_8.2_CLIENTS2/NBClients/anb/Clients/usr/openv/netbackup/client/Linux/RedHat2.6.32/cp_to_client

File /usr/openv/tmp/install_trace.29440 contains a trace of this install.

That file can be deleted after you are sure the install was successful.

I notice it says it can't determine the certificate status of either master or local host, but I can definetly see the certs from both client and master with no issues.

[root@<client name> NetBackup_8.2_CLIENTS2]# /usr/openv/netbackup/bin/nbcertcmd -listCACertDetails

Subject Name : /CN=nbatd/OU=<master server>/O=vx

Start Date : Jun 27 12:24:09 2019 GMT

Expiry Date : Jun 22 13:39:09 2039 GMT

SHA1 Fingerprint : <finger print>

 

[root@<client name> netbackup]# /usr/openv/netbackup/bin/nbcertcmd -getCertificate

Host certificate and certificate revocation list already exist for master server [<master server>]

[root@<client name> netbackup]# /usr/openv/netbackup/bin/nbcertcmd -listcertdetails

Master Server : <master server>

Host ID : <host id>

Issued By : /CN=broker/OU=<master server>/O=vx

Serial Number : <serial number>

Expiry Date : Feb 06 15:56:40 2021 GMT

SHA1 Fingerprint : <finger print>

Operation completed successfully.

  • Krutons's avatar
    Krutons
    5 years ago

    Genericus  I have plenty of free space, that wasn't the issue.

     

    So I manually removed the RPM packages for everything after stopping the services. Then reran the install and it completed successfully with no issues.

     

    [root@<client name> NetBackup_8.2_CLIENTS2]# rpm -qa | grep VRTS

    VRTSnbcfg-8.2.0.0-1561464060.x86_64

    VRTSpbx-1.11.17.1-1.x86_64

    VRTSnbpck-8.2.0.0-1561464247.x86_64

    VRTSnbjava-8.2.0.0-1561464074.x86_64

    VRTSnbjre-8.2.0.0-1561464061.x86_64

    VRTSnbclt-8.2.0.0-1561464188.x86_64

    VRTSpddea-12.0.0.0-0140.x86_64

  • DPFreelance's avatar
    DPFreelance
    Level 4
    5 years ago

    Looks like it failed installing PBX?  (I've never seen an install fail on that)

    "Failed to install VRTSpbx status = 1."

     

    For certificate stuff, have you tried using a re-issue certificate?  Or using the -force option?  Or deleteing all the certs (plus root CA) and re-acquiring them?

  • X2's avatar
    X2
    Moderator
    5 years ago

    I had one Linux (RHEL 6) client which failed with 7226. Normally, 7226 came up with Windows servers not having enough free space. But the RHEL 6 issue was peculiar (VxUpdate failed and so did manual install). In the end, it ended out to be non-NetBackup issue. The rpm database on the machine was corrupt. Once I recreated it, the installation (manual as there was no working NetBackup on the machine) went smoothly.

    Also, as your servers shows VRTSnbpck as version 8.2, doesn't hurt recreating the rpm database and verifying which packages are actually installed.

    • Krutons's avatar
      Krutons
      Moderator
      5 years ago

      DPFreelance I have deleted all the certs on the client and re-deployed, same result.

      X2 I'll give that a try, didn't think about that. Even though I am still able to query the rpm packages, it could still be corrupt? Is there a way to verify that or check before I go and try to rebuild the rpm database.

      • davidmoline's avatar
        davidmoline
        Level 6
        5 years ago

        Krutons the other thing that is showing as a problem is the failure to remove the previous PBX package. You may want to see if you can manually remove that package from one of the systems. It could be a RPM DB corruption as suggested, or there could be something else (is PBX running in secure mode - although why that would cause a failure I don't know). Has the pbx_exchange process stopped (can it be killed)?

        Removing the PBX package manually will allow the new install (however you execute it) to just proceed without the removal step (so should allow the client package to install - unless there are other issues such as the rpm db corruption).

        I'd be surprised if this was a cert related issue - it looks to me a more fundamental OS rpm uninstall.install problem.