cancel
Showing results for 
Search instead for 
Did you mean: 

Big issue on the activity monitor with 7.5.0.5

Fabrice_P_
Level 4
Certified

Hello,

We installed 7.5.0.5 on last friday and since we have a lot of issues regarding the "Activity Monitor" which seems to be inconsistent. 

Basically, we have a lot of backup jobs (more than 200) that are either queued, active or "unknown". If we look more closely to the details, we can see that the jobs are actually finished.

  • We can see the info "done, status 0" but the job is stuck at the "validating image for client xxx".
  • The backup somehow successful because it is validated in the catalog and we can restore data from it.
  • OPSCenter do not see those jobs at all.

Also, if I check "Report/Problems" on the console we see a lot of "socket open failed" critical errors. It seems that some nbu process fail to communicate between each other.

We did not had any issue of this type before this 7.5.0.5 (we were in 7.5.0.4).

I logged a support case but it was more than 2 days ago and was supposed to be contacted 2h later.

...I don't have any answer yet.

 

Regards,

50 REPLIES 50

Marianne
Level 6
Partner    VIP    Accredited Certified

Hopefully a TN will be created?

This post (and the (small??) amount of users who said that they had same problem) was enough to create wide-spread panic.

CRZ
Level 6
Employee Accredited Certified

Wide-spread panic?  A small (and vocal) group of users posted about this issue here, in addition to opening Support cases.  Support and Engineering followed their processes (which, admittedly, never seem to go fast enough when YOU'RE the one experiencing the problem) and through those processes, we're close to being able to say that we know with certainty what this issue is, how it arises and how to resolve it.

Keep in mind that all the indications we've had thus far are that despite how they are appearing in the Activity Monitor, the backups in question SUCCEEDED.  No data has been lost.  NetBackup is not indicating that your systems are protected when they're not - in fact, you could interpret it as the exact opposite - we're indicating that your system may not be protected when in fact it is!  And, as has been said previously in this thread, a very very large majority of the folks who have applied 7.5.0.5 haven't seen this issue at all.  As problems go...well, we'd prefer to have zero problems, obviously, but this one - while aggravating to those experiencing it - is rather tame when you place it on the "catastrophe" scale.  I'm not trying to make any excuses, and I much would have preferred to say "7.5.0.5 is defect free!" but I think we all knew I was never going to be able to say that despite all our best efforts prior to this release.  What's the saying?  "Stuff happens."  This sure happened!  It MAY have also hit the fan as well.  We're working hard to fix it AND get you feeling better.

Of course, we will release a TechNote - but we can't do it until we have all of the information.  We want to be sure we've fixed it before we tell you we've fixed it!  We also want to be sure we've narrowed the scope as much as we can, because as has been noted, this only affected a small number of folks who have applied 7.5.0.5.  We want to be able to provide a specific set of conditions so you can quickly determine if you might be affected (or not!) and if you'd need to call us for the EEB.

When the TechNote is released, the Late Breaking News will also be updated to point to the document.  I'm absolutely sure someone will let you know in this thread as well.  I don't want to give any specific dates on WHEN that'll happen, but know that it will be as soon as humanly possible.

Finally, you've probably already figured this out, but we're not pulling 7.5.0.5 over this issue.  Some people may find they'll need to install an EEB after they upgrade to 7.5.0.5.  Some people may want to wait on a 7.5.0.5 upgrade until we announce the availability of that EEB.  Some folks may apply 7.5.0.5 anyway and discover that they're not affected.  (Most folks, I hope!)  Some folks may apply 7.5.0.5, find their Activity Monitor IS affected, and live with it until they can call us up for an EEB.  And some folks will do something completely different which I haven't even dreamed up, because that's how the world works.  :)

What I WILL say is the ONLY way to get a defect addressed is to open a Support case and get Symantec working on it.  You can post here - and posting here is a great way to see if other people are sharing your trouble, and if they ARE, to get more people aware there are some cases that need to be resolved - but if you don't have that Support case, Connect alone will probably not get you help if you're experiencing a well and true defect in NetBackup.  This is probably obvious to everybody, but I add it here because there are some folks who try to use Connect as a substitute for our technical support - and sometimes you can even get away with that, but the truth is while Connect is great for some stuff, it should only be a SUPPLEMENT to a true tech support case when you have a "real" issue.  Let us help you!

TimWillingham
Level 5

Wow, just wow!

Fabrice_P_
Level 4
Certified

I can confirm this morning that the issue is completely gone after the installation of the EEB. smiley

Thanks a lot for the reactivity and good job !

HEMANPR
Level 6

Hello

I have the same issue. Where I can download this EBB?

Thanks

Fabrice_P_
Level 4
Certified

For the time being, you have to open a support ticket.

dthor
Level 3

Does anyone have the EEB number I have opened up a case for my preperation on the upgrade to

7.5.0.5

JMCarr
Level 0

EEB# is 3106719 

CRZ
Level 6
Employee Accredited Certified

Followup!

A TechNote AND hotfix are now available for this issue.

After upgrading a master server to NetBackup 7.5.0.5 (or applying NetBackup 5200/5220 Appliances 2.5.2 to an Appliance running as a master server), the Activity Monitor reports inconsistent information on job status for some jobs. Additionally, some jobs may not display in OpsCenter.
 http://symantec.com/docs/TECH203521

The short version is the Java GUI wasn't happy when it encountered a filename that contained an apostrophe/single quote (').

Our apologies to the early adopters who ended up discovering this defect!  But, happily, we now have this hotfix available to work around the issue.

The LBN has been updated with a link to this doc (may take some time for the cache to clear if you don't see it right now):
 http://symantec.com/docs/TECH178334

Ray_Esperanzate
Level 4

Just want to add that I am experiencing this issue as well, where the jobs are showing as active and stuck at the "validating backup image" step after going to 7.5.0.5.  It's quite random and doesn't seem to happen on the same clients all the time.  Going to open up a case now to get that EEB.

 

EDIT :  Looks like Chris posted something as i was typing my comment :)  Off to download and apply the EEB.

HEMANPR
Level 6

Thanks CRZ