02-19-2016 08:36 AM
Good Morning-
All of my media servers exist at different customer locations (mostly in the US, a few overseas) and are all part of the same WAN link, which has its moments. Sometimes things are great, sometimes, well, not so much.
What I am finding is, when this link is a little slow, even for a single media server, the entire Java console becomes very lagged and takes forever to open an element. Example: Today, when I click on Media, while one of the media servers is running a little slow, it takes forever to expand the items below it. I can watch in the command prompt window behind the Java console and see it sitting on a particular server for a long time, then finally says it was unsuccessful.
Is there a timeout settings somewhere I can change? It almost seems like NBU is waiting 5 minutes to hear back from a server, then moving on to the next one.
I believe this also brings up the topic of an offline media server -- what is the typical practice when a media server goes offline? Let's say the power goes out at a facility and the server goes down, this also makes the console operate extremely slow while NBU attempts to contact the host. This can happen at any time, depending on the site -- the only solution I could come up with so far is to ping the host via a crontab and when it fails, mark the host admin offline. Does this make any sense?
Thoughts?
JustN
Solved! Go to Solution.
02-23-2016 06:14 AM
I've noticed two time out variables used by the Java Admin Console.
See NBJAVA_CORBA_DEFAULT_TIMEOUT and NBJAVA_CORBA_LONG_TIMEOUT - I'm looking at the v7.7.2 Admin Guide I right now, but I guess they'll also be documented in the admin guide for your version. Maybe reducing these will result in a quicker drop/release of element sthat get stuck/hung/waiting in your instance of admin console when devices are no longer reachable.
02-19-2016 09:47 AM
02-19-2016 11:21 AM
Pure guess this... Java Console, View, Options..., Devices (tab), and select "Show connections only for selected devices". Dunno if it will help at the media server level. Worth a try?
.
Re:
"I believe this also brings up the topic of an offline media server -- what is the typical practice when a media server goes offline? Let's say the power goes out at a facility and the server goes down, this also makes the console operate extremely slow while NBU attempts to contact the host. This can happen at any time, depending on the site -- the only solution I could come up with so far is to ping the host via a crontab and when it fails, mark the host admin offline. Does this make any sense?"
...sounds like you're doing the right thing. But, what does "mark the host admin offline" mean?
02-19-2016 11:28 AM
What the java console setting of:
FORCE_IPADDR_LOOKUP=? <<< BTW - I've put a question mark in here :)
...for your host that you are running the java admin console on?
02-19-2016 12:29 PM
Configuring independent masters... Interesting thought... But, that would require me to manage each one of the masters independently, no? I would loose the ability to manage policies, etc... all directly from within a single Java console?
02-19-2016 12:32 PM
> ...sounds like you're doing the right thing. But, what does "mark the host admin offline" mean?
The administration guide talks about a process through NBEMMCMD to mark a media server as administratively down, such that if a media server were going to be offline for long enough, you could tell NBU to ignore it. At least, that's the way the guide talks.
My thought was, to ping the remote host every 5 minutes or so and, if the host was unreachable, mark it as administratively down... And if was up, issue the command to mark it online.
I haven't implemented this, but it was a thought.
02-19-2016 12:33 PM
I am not sure how this would help? The NBU server itself has all of the IP's in a host file (not using DNS) and the lookups seem to be OK, it's just a timeout trying to talk to the remote media server. I'm interested, if you can explain more?
02-19-2016 12:34 PM
Ok, let's ignore the IP lookup angle.
02-19-2016 12:36 PM
That's either exactly what I would have suggested, or very similar, let me check commands... brb.
02-19-2016 12:40 PM
This is what I was thinking: ./nbemmcmd -updatehost -machinename MEDIA SERVER NAME -machinestateop set_admin_pause -machinetype media -masterserver MASTER SERVER
02-19-2016 12:47 PM
Found it.
I used to use this in scripts to de-activate media servers when doing maintenance:
vmoprcmd -hoststatus -h med99-local vmoprcmd -deactivate_host -h med99-local vmoprcmd -hoststatus -h med99-local
...and in DOS/batch test %errorlevel% after each command...
...or in shell scripts test $? after each command.
02-19-2016 12:47 PM
Maybe:
a) either will work
b) neither will work
c) both required
?
02-19-2016 12:52 PM
Your command looks a bit more serious though. ;)
.
I think the vpoprcmd just stops new work being assigned to the media server.
Whereas your command looks like it do what you want.
If it was me, I'd take your advice, and try the admin pause.
02-19-2016 06:20 PM
Check the technote for more explaination
https://www.veritas.com/support/en_US/article.TECH77811
02-22-2016 01:06 PM
Fairly sure both of your commands do the same thing, I think this is also the same as right clicking on the media server in the GUI and selecting Deactivate.
02-22-2016 03:02 PM
Yeah, I'm sure the vmoprcmd command is the same as you say, but I think that the nbemmcmd with -machinestateop set_admin_pause goes a step further in forcing admin consoles to ignore a server. I need to test it, if I get time - as I wasn't aware of that command option before.
02-23-2016 06:14 AM
I've noticed two time out variables used by the Java Admin Console.
See NBJAVA_CORBA_DEFAULT_TIMEOUT and NBJAVA_CORBA_LONG_TIMEOUT - I'm looking at the v7.7.2 Admin Guide I right now, but I guess they'll also be documented in the admin guide for your version. Maybe reducing these will result in a quicker drop/release of element sthat get stuck/hung/waiting in your instance of admin console when devices are no longer reachable.