04-26-2017 05:19 PM
Having an issue with a media server I'm trying to upgrade from 2.7.3 to 3.0. Failing with error:
"NetBackup jobs are currently in progress. Stop all NetBackup jobs and then try the upgrade again. The software upgrade has been aborted!"
Funny thing is... this is a new appliance and is not in production yet... so there are *never* jobs running on it.
And when I do a "bpps -x" there are no jobs running that I can see:
# bpps -x
NB Processes
------------
root 73486 1 0 18:13 ? 00:00:00 /usr/openv/netbackup/bin/private/nbatd -c /usr/openv/var/global/vxss/eab/data
root 73497 1 0 18:13 ? 00:00:00 /usr/openv/netbackup/bin/vnetd -standalone
root 73505 1 0 18:13 ? 00:00:00 /usr/openv/netbackup/bin/bpcd -standalone
root 73565 1 0 18:13 ? 00:00:00 /usr/openv/netbackup/bin/nbdisco
root 73717 1 0 18:13 ? 00:00:00 /usr/openv/pdde/pdcr/bin/spad
root 73933 1 47 18:13 ? 00:01:32 /usr/openv/pdde/pdcr/bin/spoold
root 74255 1 0 18:14 ? 00:00:00 /usr/openv/netbackup/bin/nbrmms
root 74310 1 0 18:14 ? 00:00:00 /usr/openv/netbackup/bin/nbsl
root 74391 1 0 18:14 ? 00:00:00 /usr/openv/netbackup/bin/nbsvcmon
root 75547 74255 0 18:14 ? 00:00:00 /usr/openv/netbackup/bin/admincmd/bpstsinfo -DPSPROXY
root 75548 74255 0 18:14 ? 00:00:00 /usr/openv/netbackup/bin/admincmd/bpstsinfo -DPSPROXY
MM Processes
------------
root 74108 1 0 18:13 ? 00:00:00 /usr/openv/volmgr/bin/ltid
root 74139 1 0 18:13 ? 00:00:00 vmd
root 74313 74108 0 18:14 ? 00:00:00 tldd
root 74386 74108 0 18:14 ? 00:00:00 avrd
Shared Symantec Processes
-------------------------
root 49479 1 0 18:12 ? 00:00:00 /opt/VRTSpbx/bin/pbx_exchange
Solved! Go to Solution.
11-03-2017 08:43 AM - edited 11-03-2017 08:46 AM
It shows this behaviour every now and then. Try again or do a reboot, give it time to "settle" and try again. The appliance runs a test job before starting the upgrade. Sometimes it's to fast, and the testjob backup, restore, expire havn't finished yet.
04-26-2017 06:17 PM
Hi Elanmbx,
I did a bit of a search regarding your issue and came up with several hits, but the one below is the only one that seemed possibly relevant to you:
https://www.veritas.com/support/en_US/article.000024704
I know the article references versions earlier than yours and you say the appliance is new and not in Prod, and never has any jobs running - but just wondering if it may have some old images on it as discussed in this article.
Hope this helps,
Steve
04-26-2017 07:03 PM
Yup - already been through that Tech Note. No help in this case...
04-26-2017 07:23 PM
Grrrr... frikkin' mystery now. I decided to bump up the logging to see if I could get further info on the error:
vcfgxlogcfg -a -p 409 -o 8 -s DiagnosticLevel=6
vcfgxlogcfg -a -p 409 -o 8 -s DebugLevel=6
Ran "Install" from the CLISH to generate the error... and this time it worked. Upgrade is presently progressing apparently normally.
I have no idea what changed that would now allow the upgrade to proceed.
But it is proceeding, so I will just get on with my day. :)
04-26-2017 07:30 PM
Hi Elanmbx,
At least it's working now - one less problem for you to deal with today
My only other suggestion without going to support would have been a reboot to ensure all processes are killed - have seen some upgrades bork, but a reboot allowed them to continue.
Hope you've bumped your logging down, so that doen't blow something up
Steve
04-27-2017 07:13 AM
Yes - part of our "standard operating procedure" is to reboot the appliance immediately prior to running the upgrade scripts. In fact, I had tried rebooting *3* times in hopes that it might somehow help...
And I did turn the logging level down. Such an easy one to forget and cause yourself EXTRA problems, eh?
04-27-2017 11:47 AM
Grrr... appliance upgrade this morning ran fine... now I have a *3rd* media server and it is exhibiting this nonsense about "NetBackup jobs are currently in progress."
THERE ARE NO JOBS IN PROGRESS.
/rantoff
05-04-2017 08:47 AM - edited 05-04-2017 08:51 AM
This continues to be quite an irritation. And I can't figure out WHY it thinks jobs are still running...
If the log pointed out the process that is causing the issue, at least that would be helpful.
Sometimes another reboot fixes this. Sometimes just a restart of NBU services. Sometimes I think it just randomly goes away.
For the record - even if you STOP all NBU processes and run the upgrade it can complain about "jobs still running"
Grrrrrrrrr...
05-05-2017 06:25 AM
elanmbx
Have you opened a case on this? I haven't heard of this from anyone else, but if it's happening across multiple systems I'm wondering if there is something in your environment that is uniquely creating an issue for the upgrade checker and needs Veritas attention.
05-09-2017 07:21 AM
No - I did not. And I eventually was able to get the upgrades completed. Several required a double reboot before I was able to get the upgrades rolling.
05-11-2017 05:05 AM
you can view the appliance log from /log folder and find out that what was the root cause earlier and why it worked now.
The logs name would be patch_output_3.0_date
05-17-2017 08:41 AM
Sadly, this does *not* work - all of these failures occur well before any upgrade logging gets written to those patch_update* log files.
Still a mystery...
06-13-2017 12:57 PM
For future reference it decides there are jobs running by doing a bpdbjobs and pulling the data directly from the Master. So if you've got some funky job in the Activity Monitor it's still totally possible to have something "running" even though all daemons are down on the appliance in question.
Now if you don't mind playing with code (and potentially getting support annoyed with you)....check out /opt/NBUAppliance/scripts/patch.pl line 3213 for the chunk looking for running jobs; I take no responsibility for anything that gets broken doing that though. =)
11-03-2017 08:43 AM - edited 11-03-2017 08:46 AM
It shows this behaviour every now and then. Try again or do a reboot, give it time to "settle" and try again. The appliance runs a test job before starting the upgrade. Sometimes it's to fast, and the testjob backup, restore, expire havn't finished yet.