Forum Discussion

imrankhan3999's avatar
8 years ago

netbackup upgradation form 2.7 to 3.0 is hung

need assiatance to fix this i am in middle of the upgrade:

hung status below:

The target version is: 3.0

Command cleanup_unused_initrd is completed.The upgrade is 2% completed.

 

need assiatance.

 

 

  • thanks for reply.

    i have found solution in other means which really worked.please find the steps below:

    Troubleshooting
    ===================
    + Attempting to check progress shows 2%, and also a strange "Command cleanup_unused_initrd" message:
    name456.Software> List version
    Appliance Version: 2.7.2
    NetBackup Version: 7.7.2


    name456.Software> UpgradeStatus

    The target version is: 3.0

    Command cleanup_unused_initrd is completed.The upgrade is 2% completed.

    ---+ Attempt to elevate, and this indicates SDCSS Util Daemon is not reachable:
    maintenance-!> /opt/Symantec/sdcssagent/IPS/sisipsoverride.sh
    Symantec Data Center Security Server Policy Override

    Unable to communicate with the SDCSS Util Daemon.


    + IPMI appears to be configured:
    name456.Support> IPMI Network Show


    ---+ Run "IPMI Network Reset," IPMI seems to be working now.
    ---+ Launch JViewer.
    + Try stopping InfraServices, as well as Processes > AdminConsole services.

    + Reboot Appliance.
    ---+ Follow 000100628 to disable SDCSS.

    + Reboot continues; rollback appears to initiate.
    ---+ Some potential errors when rolling back to MSDP storage checkpoints, e.g.:
    V-3-24623: storage checkpoint list failed onl, Permission denied (13)


    + Rollback completes:
    name456.Software> List Version
    Appliance Version: 2.7.2
    NetBackup Version: 7.7.2


    The appliance version is 2.7.2 and not in upgrade state.

    ---+ Remove old pre-upgrade checkpoint.
    name456:/home/maintenance # lvscan


    ---+ Checked patch_log_3.0.
    -----+ Long repeating messages waiting for bptm to terminate:


    Looking for NetBackup processes that need to be terminated.
    Stopping nbsvcmon...

    There may be backups and/or restores active.
    They will be terminated....
    Killing bptm processes...
    Waiting for active bptm processes to terminate
    Waiting for active bptm processes to terminate
    Waiting for active bptm processes to terminate
    Waiting for active bptm processes to terminate
    Waiting for active bptm processes to terminate
    Waiting for active bptm processes to terminate
    Waiting for active bptm processes to terminate
    ...

    ---+ Set media server to administrative pause to prevent jobs from starting during upgrade:
    name456:/log # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop set_admin_pause -masterserver name123
    NBEMMCMD, Version: 7.7.2
    Command completed successfully.

    + Retry upgrade to 3.0.
    ---+ Eventually reaches 1%, but several minutes elapse before the new patch_log_3.0_* file appears.
    ---+ Seems to get past the stop_nbu subroutine this time.
    + Reaches 13%.

    + Eventually reaches post-upgrade checkpoint.

    ---+ Need to clear admin pause:
    # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop clr_admin_pause -masterserver name123

    ===================
    References
    ===================
    How to skip the backup/restore portion of the self_test when upgrading NetBackup 52xx appliances.
        Article:000070876
        Publish:October 21, 2015
        Article URL:http://www.veritas.com/docs/000070876

    CAUSE:
    ------------------------------
    + Initial upgrade failed due to hanging bptm processes that could not be terminated.

    SOLUTION/WORKAROUND:
    ------------------------------
    + Remove previous pre-upgrade checkpoint:
    name456:/home/maintenance # lvscan


    name456 :/home/maintenance # ls -la /repository/checkpoints/


    ---+ Place media server in Administrative Pause:
    name456:/log # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop set_admin_pause -masterserver name123
    NBEMMCMD, Version: 7.7.2
    Command completed successfully.

    + Retry upgrade to 3.0.
    ---+ Eventually reaches 1%, but several minutes elapse before the new patch_log_3.0_* file appears.
    ---+ Seems to get past the stop_nbu subroutine this time.
    + Reaches 13%.

    + Eventually reaches post-upgrade checkpoint.
    + Customer will validate NetBackup is working.
    ---+ Need to clear admin pause:
    # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop clr_admin_pause -masterserver  name123

2 Replies

  • go to the maintainance mode from clish  cd to  /log and there you will see patch logs  it would showing the information why its hanged ? Also review the appliance self test was successfull ?  

     

     

    • imrankhan3999's avatar
      imrankhan3999
      Level 2

      thanks for reply.

      i have found solution in other means which really worked.please find the steps below:

      Troubleshooting
      ===================
      + Attempting to check progress shows 2%, and also a strange "Command cleanup_unused_initrd" message:
      name456.Software> List version
      Appliance Version: 2.7.2
      NetBackup Version: 7.7.2


      name456.Software> UpgradeStatus

      The target version is: 3.0

      Command cleanup_unused_initrd is completed.The upgrade is 2% completed.

      ---+ Attempt to elevate, and this indicates SDCSS Util Daemon is not reachable:
      maintenance-!> /opt/Symantec/sdcssagent/IPS/sisipsoverride.sh
      Symantec Data Center Security Server Policy Override

      Unable to communicate with the SDCSS Util Daemon.


      + IPMI appears to be configured:
      name456.Support> IPMI Network Show


      ---+ Run "IPMI Network Reset," IPMI seems to be working now.
      ---+ Launch JViewer.
      + Try stopping InfraServices, as well as Processes > AdminConsole services.

      + Reboot Appliance.
      ---+ Follow 000100628 to disable SDCSS.

      + Reboot continues; rollback appears to initiate.
      ---+ Some potential errors when rolling back to MSDP storage checkpoints, e.g.:
      V-3-24623: storage checkpoint list failed onl, Permission denied (13)


      + Rollback completes:
      name456.Software> List Version
      Appliance Version: 2.7.2
      NetBackup Version: 7.7.2


      The appliance version is 2.7.2 and not in upgrade state.

      ---+ Remove old pre-upgrade checkpoint.
      name456:/home/maintenance # lvscan


      ---+ Checked patch_log_3.0.
      -----+ Long repeating messages waiting for bptm to terminate:


      Looking for NetBackup processes that need to be terminated.
      Stopping nbsvcmon...

      There may be backups and/or restores active.
      They will be terminated....
      Killing bptm processes...
      Waiting for active bptm processes to terminate
      Waiting for active bptm processes to terminate
      Waiting for active bptm processes to terminate
      Waiting for active bptm processes to terminate
      Waiting for active bptm processes to terminate
      Waiting for active bptm processes to terminate
      Waiting for active bptm processes to terminate
      ...

      ---+ Set media server to administrative pause to prevent jobs from starting during upgrade:
      name456:/log # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop set_admin_pause -masterserver name123
      NBEMMCMD, Version: 7.7.2
      Command completed successfully.

      + Retry upgrade to 3.0.
      ---+ Eventually reaches 1%, but several minutes elapse before the new patch_log_3.0_* file appears.
      ---+ Seems to get past the stop_nbu subroutine this time.
      + Reaches 13%.

      + Eventually reaches post-upgrade checkpoint.

      ---+ Need to clear admin pause:
      # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop clr_admin_pause -masterserver name123

      ===================
      References
      ===================
      How to skip the backup/restore portion of the self_test when upgrading NetBackup 52xx appliances.
          Article:000070876
          Publish:October 21, 2015
          Article URL:http://www.veritas.com/docs/000070876

      CAUSE:
      ------------------------------
      + Initial upgrade failed due to hanging bptm processes that could not be terminated.

      SOLUTION/WORKAROUND:
      ------------------------------
      + Remove previous pre-upgrade checkpoint:
      name456:/home/maintenance # lvscan


      name456 :/home/maintenance # ls -la /repository/checkpoints/


      ---+ Place media server in Administrative Pause:
      name456:/log # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop set_admin_pause -masterserver name123
      NBEMMCMD, Version: 7.7.2
      Command completed successfully.

      + Retry upgrade to 3.0.
      ---+ Eventually reaches 1%, but several minutes elapse before the new patch_log_3.0_* file appears.
      ---+ Seems to get past the stop_nbu subroutine this time.
      + Reaches 13%.

      + Eventually reaches post-upgrade checkpoint.
      + Customer will validate NetBackup is working.
      ---+ Need to clear admin pause:
      # nbemmcmd -updatehost -machinename name456 -machinetype media -machinestateop clr_admin_pause -masterserver  name123