06-16-2020 12:28 AM
I have Netbackup running on Windows 2012 and the console is freezing as shown in the attachment, please advise.
We performed the following steps but still the error persists
On the backup server edit the install_path\VERITAS\Java\setconf.bat as follows:
Modify this default setting.
SET NBJAVA_CORBA_DEFAULT_TIMEOUT=60
Change it to:
SET NBJAVA_CORBA_DEFAULT_TIMEOUT=300
06-16-2020 03:25 AM
It is not very likely you can recover this situation - it may be the daemon not responding, and not a java issue.
The fastest method to get back in normal operation may be a re-start of the Netbackup services on the master server.
06-16-2020 04:14 AM
Hello,
I think it's better to restart Netbackup services in the master server.
06-16-2020 08:07 AM
Well, This could be due to a number of issues.
Bouncing of NBU Services generally fixes this . Unless its something else.
06-18-2020 07:09 AM
Bouncing of NBU Services means what do I need to do?
06-18-2020 10:33 AM
This mean you have to stop NBU
bpdown -f -v
and start it back again
bpup -f -v
that's all.
06-21-2020 07:38 AM
Please find attached the command execution results and advice.
06-21-2020 12:55 PM
Hey
Something is NOK within your NBU domain... These four are really needed processes:
> NetBackup Database Manager -- NOT STARTED
> NetBackup Request Daemon
> NetBackup Request Daemon -- NOT STARTED
> NetBackup Job Manager
> NetBackup Job Manager -- NOT STARTED
> NetBackup Policy Execution Manager
> NetBackup Policy Execution Manager -- NOT STARTED
Check event viewer to figure out why these were having trouble to start. Also create logging for these processes in
install_dir\Netbackup\logs\ for
bpdbm, bprd, nbjm and nbpem - set verbose logging for whole NBU services... and search for any clue out there...
here is some link for nbu logging reference guide
06-23-2020 12:22 AM
I see masive applciation events as below
EMM interface initialization failed, status = 334
Systems Logs:
The IPMI device driver attempted to communicate with the IPMI BMC device during normal operation. However the communication failed due to a timeout. You can increase the timeouts associated with the IPMI device driver.
06-23-2020 01:07 AM
Please have another look at @quebek 's last post. We need to see logs to understand what is happening here and why services are not starting.
Has anyone perhaps change the master server's hostname?
Or changed SERVER entries in Host Properties or Registry so that master server name is no longer at the top of the server list?
The logs that I am interested in are: bpdbm, bprd and nbemm.
Please do the following:
Check and confirm that bpdbm and bprd folders exist in <install_dir>Veritas\Netbackup\logs\.
If not, create these folders.
Stop NBU. Check with bpps that all processes/services have stopped.
Start NetBackup again.
If the same NOT STARTED messages are seen, please collect logs and upload here.
To collect EMM log, please run the following command (which will collect emm events for the last 10 minutes - assuming you started NBU within the last 10 minutes):
vxlogview -o 111 -t 00:10:00 > C:\Temp\emm.txt
(Assuming C:\Temp exists. Create the folder or use another temp directory that exists.)
Copy newest log file in bprd and bpdbm folders to bprd.txt and bpdbm.txt.
Please upload .txt files.
06-23-2020 02:07 AM
We have not changed the hostname.
"changed SERVER entries in Host Properties or Registry so that master server name is no longer at the top of the server list?"
Please explain how to trace the server list order.
06-23-2020 02:47 AM
If Server entries are updated via Host Properties, it will update the Registry.
From Host Properties, it is more difficult to make a mistake, because the Master server entry is clearly marked.
When Server entries are added by editing the Registry, the Server entries are all in the same line. Only difference is that the Master name must be first.
Please collect the logs that I have listed?
We should get a good indication of what is wrong.
06-23-2020 08:02 AM
Please find attached the requried logs
06-23-2020 08:45 AM
I do not see anything in bprd or bpdbm logs that looks remotely useful. They do not even appear to be legacy logs like I'm expecting...
In EMM log we can see that there was no hostname change, but something seems to be wrong with NBU databases.
EMM starts and then terminates again. This is probably preventing bpdbm and bprd to start.
Could you please locate the server.log file in <install_path>\NetBackupDB\log\ folder?
Please copy to server.txt and upload here.
06-24-2020 12:08 AM
Please find attached server log file
06-24-2020 01:01 AM
I do not see any errors in server log. There are only warnings about fragmentation, but no real issues.
I am confused about the logs files that you have in bprd and bpdbm folders - the bprd and bpdbm files look like copies of the emm log. Please have another look in these folders on the master?
In bpdbm folder, we see ALL_ADMINS log that looks like a legacy log. It simply repeats these 3 lines over and over:
17:25:31.648 [4980.4016] <2> read_legacy_touch_file: Found C:\Program Files\Veritas\NetBackupDB\data\vxdbms.conf; requested from (vxdbms_conf.cpp.1529).
17:25:31.662 [4980.4016] <16> DbmOdbcConnect::initialize_odbc: Odbc connection failed ErrMsg [Sybase][ODBC Driver][SQL Anywhere]Invalid user ID or password, ErrCode -1, Sqlstate 28000, ConnStr DRIVER=NB SQL Anywhere;PROWS=1000;PBUF=8m;CS=utf8;UID=DBM_MAIN;PWD=***************;LINKS=shmem,tcpip{PORT=13785};ServerName=NB_BRS001.MYDOMAIN.NET;DBN=NBDB;CON=DBM_4980_3
17:25:31.668 [4980.4016] <32> UpgradeHelper::perform_upgrade: Cannot connect to the database, please ensure services are properly started.
I cannot tell you what is causing this or how to fix it.
What you really need is to log a call with Veritas Support and have a knowledgeable support engineer use webex to look around, but I guess you do not have active maintenance and that is why you are still on version 7.x?
Do you perhaps have a support agreement with a partner whou could possibly try to assist?
You can try to follow steps in this TN to rebuild vxdbms.conf:
https://www.veritas.com/content/support/en_US/article.100021292
but I cannot guarantee that this will fix your issue.
Still worth a try IMHO.
06-24-2020 03:31 AM
I have decided to re-install the software. Shall I re-install (in place install) or perform a complete uninstall and re-install.
Do I need to re-deploy the agents if I have to perform a complete fresh install of Netbackup?
06-24-2020 06:37 AM
That seems to be very extreme.
I would personally try to fix it so as not to lose backup history.
For a re-install, you will completely uninstall NBU, patch to same level as before, then perform NBU catalog recovery.
For how long has NBU be running in this environment?
Do you have a recent good catalog backup?
Without a catalog backup to recover, you will need to start from scratch and start to import backup images from tapes or disk.