Problem with Media and Device Management in 6.5.6

[kenrblan]

I have Netbackup 6.5.6 Master/Media Server running on Windows 2003. In the last few days, I have started getting the error "Error connecting to Media Manager on ops078-m17: network protocol error(39)." when clicking to expand Media in the GUI. When I try to expand Devices, I get the error "Unable to obtain the device list from the Enterprise Media Manager server. Database Server is down(93)."  If I look at Activity Monitor, backup jobs are processing correctly. My Vault Jobs are running correctly and are properly managing tape ejections and imports.

I use the GUI frequently to identify tapes that have been frozen or suspended, but have not been able to view these through the GUI. All of the normal Netbackup services appear to be running. I am running the GUI on the Netbackup Master server when seeing these errors. I have already tried rebooting the Master and Media servers with no effect.

Any recommendations?

[RiaanBadenhorst]

Are you opening the GUI from your desktop, i.e. is it installed on your desktop , or do you open it on the master?

[kenrblan]

This occurs when opening on the master. Reports, policies, Host Properties, etc that are grouped under Netbackup Management all work fine. Device monitor seems to work fine, but the Media and Devices generate the previously mentioned errors. When I try to use the GUI from another workstation or Media Server, the same behavior occurs.

[HEMANPR]

Commonly, error with the Netbackup Database are fixed with a simple restart. (Errors not presented or affect backups jobs) Commonly affect the EMM Operations.

Try restart your server 1st. Let us know the results.

[kenrblan]

I tried restarting Netbackup Services on both the Master and Media services with bpdown/bpup. I tried rebooting both Master and Media servers. If I hadn't tried that first, I wouldn't have posted the question (and noted the attempted restarts in in original post).

One other symptom I have observed is the inability to view/change Master Server properties by going to Host Properties->Master Servers->Right Click on master->Properties. I get the request has timed out error. If I try to open the properties by going to client properties or Media Server properties for the same host, it works like normal but doesn't display items normally associated with a Master Server.

Here are the results of a bpps -s listing:

C:\Program Files\VERITAS\NetBackup\bin>bpps -s
* OPS078-M17                                            10/06/10 10:11:49.207
COMMAND           PID      LOAD             TIME   MEM                  START
dbsrv9           1536    0.000%        27:08.343   15M  10/05/10 15:22:31.734
bpcompatd        2148    0.000%            8.734  9.6M  10/05/10 15:22:36.875
bpinetd          2216    0.000%           53.750  4.6M  10/05/10 15:22:37.875
bpjava-msvc      2500    0.000%            0.000  3.6M  10/05/10 15:22:39.453
bmrd             2612    0.000%            0.625   12M  10/05/10 15:22:40.968
bpdbm            3052    0.000%           35.843   12M  10/05/10 15:22:43.531
nbemm            3196    0.000%        32:12.390   36M  10/05/10 15:22:45.203
nbevtmgr         1988    0.000%         3:52.125   16M  10/05/10 15:22:48.000
nbjm             4128    0.000%         3:07.843   19M  10/05/10 15:22:49.000
bprd             4256    0.000%            9.000   11M  10/05/10 15:22:51.265
nbrb             4276    0.000%         2:46.421   20M  10/05/10 15:22:51.531
nbsl             4348    0.000%           41.250   20M  10/05/10 15:22:52.578
nbsvcmon         4404    0.000%            0.609   11M  10/05/10 15:22:54.265
nbstserv         4420    0.000%         6:26.750   19M  10/05/10 15:22:54.421
nbvault          4440    0.000%            0.468   14M  10/05/10 15:22:55.437
vmd              4460    0.000%            1.218   11M  10/05/10 15:22:55.921
bpjobd           4556    0.000%         1:31.093   11M  10/05/10 15:23:15.312
nbpem            4960    0.000%           37.796   22M  10/05/10 15:23:19.687
nbproxy          5384    0.000%            7.265   12M  10/05/10 15:23:25.828
nbrmms           5416    0.000%           10.140   20M  10/05/10 15:23:26.875
nbproxy          4336    0.000%           14.734   14M  10/05/10 15:23:53.171
nbproxy          5412    0.000%            7.843   12M  10/05/10 15:24:58.140
nbproxy          7600    0.000%            0.375   12M  10/05/10 15:52:40.748
bpbrmvlt         2368    0.000%            0.078  5.0M  10/06/10 04:00:05.206
vltrun           2100    0.000%            3.296   15M  10/06/10 04:00:05.315
bpduplicate      7072    0.000%            8.484   13M  10/06/10 06:19:01.351
tracker          7036    0.000%            0.406   12M  10/06/10 08:06:59.669
bpcd             7944    0.000%            0.125  6.1M  10/06/10 08:19:39.129
bpdbm            4624    0.000%            0.218   12M  10/06/10 08:19:39.426
nbproxy          2496    0.000%            0.140   11M  10/06/10 08:37:05.330
nbproxy          2504    0.000%            0.140   11M  10/06/10 08:37:57.204
nbproxy          6072    0.000%            0.156   11M  10/06/10 08:40:19.090
nbproxy          7348    0.000%            0.140   11M  10/06/10 08:40:54.354
nbproxy          6816    0.000%            0.140   11M  10/06/10 09:05:01.933
NBConsole        8164    0.000%           16.281   42M  10/06/10 09:33:00.738
bpbrm            7352    0.000%            0.265   11M  10/06/10 10:00:03.730
bpbrm            6204    0.000%            0.218   11M  10/06/10 10:04:47.549
bptm              276    6.154%           59.843   22M  10/06/10 10:05:02.376
bptm             7220    0.000%           45.468   19M  10/06/10 10:06:01.640
nbproxy          6632    0.000%            0.250   11M  10/06/10 10:06:33.280
nbproxy          2064    0.000%            0.390   11M  10/06/10 10:10:48.725
bpps             7024    1.539%            0.234  2.4M  10/06/10 10:11:47.785

[RiaanBadenhorst]

Hi,

That is odd, sounds like a type of network issue. Even though its on host, i know. Are you able to run commands like nbemmcmd etc from the master command prompt, and issues?

Do you get the same error if you open GUI on a media server and perform the same operations?

[kenrblan]

The same errors show up regardless of which system I use to open the Administration Console GUI.

I am able to run nbemmcmd at the master's command prompt without a problem. I ran "nbemmcmd -listmedia -brief -poolname Scratch" and "nbemmcmd -getemmserver."  Both commands returned the expected results. This is just weird.

[Will_Restore]

check this document for possible solution

http://www.symantec.com/business/support/index?page=content&id=TECH50318&actp=search&viewlocale=en_U...

Resolution:
The NetBackup Client Service was being started by an account other than the local system account.  The NetBackup Client Service must be started an account which has the following permissions:

 

- Act as part of the operating system

- Replace a process level token

- Logon as a service

- Create a token object

Change the permissions for the account starting the NetBackup Client Service and restart the service.  It may be necessary to wait 10-15 minutes for replication throughout the Windows environment to occur for all changes to take affect.  

[kenrblan]

Since all of my netbackup services are running as local system, I don't think this article applies. Thanks for the suggestion though.

[ashish_patil01]

By any chance have you recently decommision any media server.

[ashish_patil01]

Please also verify if you see any duplicate entries for robot under the windows device manager and if it exists please delete the duplicate entry.

[kenrblan]

Yes, I have recently decommisioned a media server. I used the nbdecommission command to do it. That was done about 1 month ago.

[kenrblan · Accepted solution]

http://www.symantec.com/docs/TECH126537

It turns out that the NB Admin Console has a default GUI connect timeout of 20 seconds. I increased that value to 60 seconds and it works now.