Delay in selecting BAR in Java Admin Console

[Eric_Engberg]

I recently took over finishing an install of a netbackup environment and am having some trouble with the Java client when selecting "Backup, Archive, and Restore.  There is a massive delay between clicking BAR and it loading.  I have access to 2 other netbackup master servers and neither have this problem.  This new install is a Windows 2008 R2 server where as the other 2 are UNIX.

It seems the delay is being caused by the Admin Console trying to make calls to some of the DB extensions.  These do not appear to be installed in the unix environments but are currently installed on the Windows environment.  I can't seem to figure out how to remove or disable these extensions.  Here are some lines from the Admin Console showing it making calls to db2 and oracle.  This environment will not need to use any DB extensions so I'd like to either disable them or better yet remove them entirely.  Hopefully without having to reinstall the master server.

"C:\Program Files\Veritas\netbackup\bin\bpubsdb2.exe" db2

(SR-1)execCommand:

        Protocol Code: 1

        Status: 104

        Time Taken: 48346ms

        Error Msg:

        Server Locale: en_US

        TO[0]: "C:\Program Files\Veritas\netbackup\bin\bpubsdb2.exe" db2

        FROM[0]:

        Aux data: null

"C:\Program Files\Veritas\netbackup\bin\bpubsora.exe" Oracle

(SR-1)execCommand:

        Protocol Code: 1

        Status: 104

        Time Taken: 48812ms

        Error Msg:

        Server Locale: en_US

        TO[0]: "C:\Program Files\Veritas\netbackup\bin\bpubsora.exe" Oracle

        FROM[0]:

        Aux data: null

[Nicolai]

This might be a issue with name resolution. Check the DNS servers respond as intended.

Pls verify the name resolution using nslookup not ping - ping has a bad habit of using WINS if DNS fails.

[Eric_Engberg]

There is no DNS setup for this network. Everything is configured via hosts file. The 2 other netbackup environments do not make these calls to the DB extensions and pull up BAR almost instantly. Each of these DB extensions being called by the GUI take about a minute to time out and return a status of 104 and it does this about 2-3 times each for DB2 and Oracle. After it finishes making these calls the GUI starts responding again and loads up the BAR frame. I'd like to just be able to disable these DB2 and Oracle calls.

[CRZ]

Are all the servers at the same version?  (What version was that, again?)

Status 104 (invalid file pathname) usually means something is missing that NetBackup is expecting, but I've never seen it take over 48 seconds to not find something here!

Could you try running these commands manually and attach the results?

"C:\Program Files\Veritas\netbackup\bin\bpubsdb2.exe" -v5 -L C:\db2log.txt db2
"C:\Program Files\Veritas\netbackup\bin\bpubsora.exe" -v5 -L C:\oralog.txt Oracle

I should warn you ahead of time there's a very good chance I'll recommend you open a support case unless it's something really blindingly obvious to you or me.

[Eric_Engberg]

Is there really no way to disable/uninstall the db2/oracle components? Neither DB2 or Oracle are installed/used in this environment. At any rate here are the requested logs.

[CRZ]

I don't think it's a "db2/oracle" issue per se, though.

Checking the logs, it's easy to find your giant pauses - they're both tripping up on something to do with uEFI, looks like, although they choked at different points:

22:12:26 INF - IsSystemBootedUsingEFIFirmware() - Enter
22:13:38 INF -  Failed to execute BcdObject.GetElement() method. hResult code: 0xd000017c

72 seconds!

22:14:46 INF - IsSystemBootedUsingEFIFirmware() - Enter
22:15:18 INF -  Failed to execute BcdStore.OpenObject() method. hResult code: 0xd000017c

"Only" 32 seconds.

I tried these same tests on my XP laptop (which also doesn't do any DB backups and also shouldn't have any EFI stuff at all...I think) and I get the same "Failed to execute" messages, but it took 3 and 0 seconds respectively.  So these informational messages aren't terribly helpful (unless the different error codes mean something to somebody in Engineering?) but the passage of time between messages definitely IS helpful.

If it were a BCD/EFI issue, you might also see weirdness when trying to back up Shadow Copy Components on this machine.  Have you seen any weirdness?  (Assuming you are doing that?)

You've posted three times in this thread but still haven't told me which version is installed.  Can you check that and report back?

Unless it turns out you're not running a supported version (in which case I'd direct you to install a supported version and try again), I'd say to open a support case, give your TSE the logs as well as the snippets you provided up top with the status 104 and link them to this thread to explain where they came from, and request an escalation to backline for more/better instructions than I can give you to troubleshoot this issue. We may need to provide you with a debug binary because the "verbose" logs I had you generate don't really seem verbose enough.  (OR they may have better advice, or even know some secret way to eliminate those particular checks that I don't.)

Sorry I don't have a real answer for you - this one is quite unusual!

EDIT: And I just noticed no clues were provided to explain the status 104, either!  So two strikes against me.  Definitely looking like it may be a support case with Engineering involvement...which is probably a giant pain for you and I apologise in advance one more time.

[Eric_Engberg]

Sorry I forgot to mention the version. I'm running 7.5.0.3 with the following EEB packs installed, NB_7.5.0.3_ET2876501_1, NB_7.5.0.3_ET2829119, and NB_7.5.0.3_ET2838857_3. I'm not doing any Shadow Volume Copy backups. Most of the environment is Red Hat Linux only the master/media server are Windows 2008 R2. The policy I made to backup the master/media server is using whatever the defaults are and backing up only SystemState:\ and the netbackup install folder.

[Will_Restore]

Multiple NICs enabled on this new Windows Master ?

[Eric_Engberg]

Yes there are multiple NICs. Two bonded together on one network and a 3rd NIC by itself on another network.

[Will_Restore]

Temporarily disable all but primary and I suspect the trouble will go away. Then you have to fix the underlying trouble (network configuration, name resolution, etc)

[Eric_Engberg]

Nope.. Tried disabling both NIC's individually and still had the same problem.  I dont' think this is a name resolution problem for 2 reasons.

1) This setup doesn't have dns.  Everything is in hosts files which are all identical.

2) The part it gets stuck on seems to deal with IsSystemBootedUsingEFIFirmware() as CRZ pointed out.

I've got an open ticket on this but haven't heard from the engineer in 2 days.  This morning claimed he would contact me today after he had lunch.  It's now 3PM EST where he is and still no word.  Hope I never need anything important from support.

[CRZ]

Hi Eric,

Have you pointed your TSE to this thread yet?  Unless he or she has some really great idea I haven't thought of, I have no idea why this wouldn't get escalated to backline right away - and then to Engineering.

[Eric_Engberg]

I have and he said he read it. I got an email from him later in the afternoon today saying he'd call me tomorrow morning now so I guess we'll see how that goes.

[Eric_Engberg]

Well I have somewhat of an update. The TSE had me let him record a session and basically just demo the problem. He pointed out there are some ODBC stuff installed and was asking why that was and what they were. One is "Backup Exec Catalog" which I agree is a little curious since Backup Exec isn't installed but more on that in a bit. The other's are 2 that start with NB (so I assume Netbackup since it is it starts with NB and says SQL Anywhere which 7.5 uses if I remember right from reading some documentation) and the 3rd is just Microsoft SQL which is probably standard for Windows 2008 Server. When clicking configure or remove on the "Backup Exec Catalog" one it gives an error which I've attached a screenshot of so I manually deleted it from the registry. Upon going back to "Backup, Archive, and Restore" in the Java GUI the "Backup Exec Catalog" ODBC comes back so it appears that is from Netbackup. The TSE has also linked me to TECH articles for things to try that referenced 6.0 or earlier versions that are the defaults in 7.5 already.

[IrememberVic20]

We also had the same problem but it would take about ten minutes to open the BAR.

And since we don't use DB2 or Oracle we just renamed the two files "bpubsora.exe" to "bpubsora.exe.old" and "bpubsdb2.exe" to "bpubsdb2.exe.old" on the server and now it take about two seconds to open.

[Eric_Engberg]

I did the same thing as a temporary fix but I'd like a real solution if possible.  I doubt renaming those would have any negative affect in our environment but I don't know for sure.  Still haven't heard anything from the TSE other than he 'is still researching this issue'.

[Mark_Solutions]

There is one other thing that i have seen have an effect - although on the Admin Console rather that the BAR GUI ... and that is when BMR has not been set up as it makes calls to that configuration and if not set up it eventually times out and the Admin Console opens.

To solve that one simply run bmrsetupmaster on the Master - or run it from the Start Menu - you don't actually have to use BMR, just have it enabled.

Can I ask why you are using the Java console when this is a Windows  System and does the same happen when opening the Windows BAR GUI?

Hope this helps

[Eric_Engberg]

The regular GUI seems to work ok but I'm not that familar with it.  The Java GUI is just want they've always used here and our support people are familar with.  Infact when this new environment was setup it didn't have the Java GUI and I was asked to install it.

The regular GUI doesn't look as straight forward to me but I haven't really  sat down and taken a look at it yet.  Maybe in the future I can try migrating to that.

I don't think BMR is our problem here as renaming bpubsdb2.exe and bpubsora.exe 'solves the problem' but maybe that info will help other people. I'll take a look just in case though.