Netbackup java administration console - activity monitor dont refreshing and zombie jobs

[frena]

Hello colleagues;

I have netbackup 6.5.5 installed on solaris10.

Java administration console (version 6.5 + 6.5.6 patch) is running from windows machine.

"Only" problem what I have is activity monitor.

1. Some of jobs which are in active status (green running man) stay in that status also after job is finished successfully (when I click onto details I  can see that job is currently done). After I start NetBackup JAC again - these jobs are normally in done status, but problems occurs after some time again with new jobs.

2. Also happens that jobs are restarted as other job (in details - job xxx was restarted as job yyy) – in activity monitor I can see only job id in such case and everything else is clear.

3. Other issue is that some of these "clear" jobs without any other note in activity monitor seems like zombie, but after click onto details it show me normally regular info about successfully job. (basically like point 1; but jobs are not in status active, but seems like zombie jobs)

Usually zombie jobs get disappeared after restart of netbackup services, but these are not usual zombie jobs.

How I wrote in point 1 some of jobs get to normal after restarting the java admin console, but these is not valid for point 2.

Any ideas how to fix it are welcome...

[Andy_Welburn]

"...
I have netbackup 6.5.5 installed on solaris10.

Java administration console (version 6.5 + 6.5.6 patch) is running from windows machine.
..."

If your master is at 6.5.5 I would drop the Admin Console to the same. It is recommended that the master is at the highest revision in the environment - it could be that this is causing the issues you are seeing.

[frena]

I dont think that this is cause of problem..

We have also another MS server with version 6.5.6 and there is the same problem...

Possible cause could be that someone is loging in via old verison of netbackup java...where can I check this?

[Andy_Welburn]

of a version that is at a higher level than the master than one with a lower version. You state yourself:
"We have also another MS server with version 6.5.6 and there is the same problem."

A few years back, I used a higher version client (don't ask me why!) & it caused no end of problems.

Have you tried dropping it to 6.5.5 (which I am still presuming is the level of your master)?

[MOHAMED_PATEL]

1.Close any open 6.x java consoles
2.Locate the Java admin console config file on the host where you are running the client from (typically a Windows workstation):

  1.Windows: <install location>\Veritas\NetBackup\java\setconf.bat
  2.Unix/Linux: /usr/openv/java/nbj.conf
3.Disable FORCE_IPADDR_LOOKUP

  1.add/change this line on Windows: SET FORCE_IPADDR_LOOKUP=0
  2.add/change this line on Unix/Linux: FORCE_IPADDER_LOOKUP=0
Note: the difference in the spelling of the parameters is correct. Also, no spaces before or after the equal sign and SET is required on Windows.

4.Restart the console and confirm the problem is resolved.

           this was a post by rusman sometime back....

[Andy_Welburn]

Thought those changes were just for performance reasons?

[MOHAMED_PATEL]

Yes, that is true - however updates should be quicker -

[Andy_Welburn]

Still think the underlying issue to be the version level, but .......

[MOHAMED_PATEL]

I also agree with the version difference - never doubted that - Vote +1

FRENA should downgrade first and test - and then try the tweak...

[MOHAMED_PATEL]

on the subject of zombie jobs, have a look at this technote:

http://www.symantec.com/docs/TECH136957

[Jaerin]

I have this exact same issue on my 7.0 environment.  It's not a GUI issue.

These aren't zombie jobs per se.  It appears that for whatever reason NBU has restarted an already existing job under a new job ID.  I haven't been able to dig down into the logs to see why it restarted it, but the result is a virtually blank job entry in the Jobs DB.  The only information on the job is the master server and a line in the detailed status that says:

"Nov 3, 2010 10:38:49 AM - job 1693289 was restarted as job 1694151"

Job ID 1694151 is a normal filesystem backup of one of my windows systems.

Only way to get rid of the jobs is to stop NBU, delete/rename the bpjobd.act.db, and restart NBU.  Essentially a work around that deletes all the job status of active jobs when there should be no active jobs.   Still doesn't answer why they are put in the activity monitor without any means of expiration.

[thesanman]

From a previous issue; jobs stuck and unable to be deleted from console:

https://www-secure.symantec.com/connect/forums/how-clear-and-delete-waiting-retry-status-50-job

Look towards the end; my final post shows the use of /usr/openv/netbackup/bin/bpjobd -r <jobid> followed by a Netbackup stop/start cycle if that helps for you.

Saves having to delete the job database.

[John_Meyers_2]

I had just recently upgraded from 6.5.4 to 7.0.1 and then about a week later went to 7.1.  I'm seeing an on-going issue of having these "blank" job-id's show up in the Activity Monitor.  I'll take everything down and clear out the active job db along with doing the bpjobd -r <jobid>, which fixes the problem for a day and then they start creeping back in again.  This is with NB 7.1 under Solaris 10 (sparc).  Is there any outstanding issues with this that I should be looking at?  I didn't have this issue under 6.5.4.

Side note that all of the blank jobids are low numbered, such as 15, 40, 48, 165, ... where as my active or real jobids are up in the 77000 range.

Thanks,

John

[thesanman]

I have the same issue after upgrading from v7.0.1 to v7.1 using RHEL 5 Master Server.  I have logged a support call.

[thesanman · Accepted solution]

There is an EEB for this.

Contact support; ask for v7.1 EEB 2380076.  You need to apply the EEB to your Master and all Media servers.