cancel
Showing results for 
Search instead for 
Did you mean: 

nbstlutil report command throwing a error "Can't obtain SLP definitions, error status = 25"

Naveenyenigandl
Level 3

When i tried to execute nbstlutil report command am getting an error "Can't obtain SLP definitions, error status = 25". Please help me to fix this issue.

17 REPLIES 17

sdo
Moderator
Moderator
Partner    VIP    Certified

Are you running as root ?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@Naveenyenigandl 

Are you running the command on the master server?

Have you checked if all required processes are running on the master?
Please show us ouput of 'bpps -x'.

Yes am running as root user and i could see all the requied services are running.

sdo
Moderator
Moderator
Partner    VIP    Certified

@Marianne asked if you are are on the master ?

yes am executing the command from Master server only

sdo
Moderator
Moderator
Partner    VIP    Certified

If you are on master, as root, and all services are running, then that report command should work.  The fact that it does not work suggests something is quite wrong with your environment.  If you are not experienced enough to setup and configure logging and to then dig through the logs on your own then I think your best bet is to open an official support case with Veritas.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@Naveenyenigandl 

I was hoping that you would show us output of bpps -x.

We need confirmation that nbstserv is running (along with the rest...)

Status 25 means that connection could not be made to a particular process on the relevant server on a port (normally 1556).
Extract from a very old TN: 
This error occurs when a Netbackup process times out while attempting to connect to another Netbackup process.
 The initial goal of troubleshooting this issue should be to identify which machines are involved with the failure.   For example, if the failure always involves the same Client or Media Server, this is where to concentrate efforts.   Once this failure point is identified, then further troubleshooting can be done to identify the specific cause.
So, something wrong with internal comms on the master server.

Have you tried to check nbstserv log on the master server?

 

@Marianne please find the below NBU services currently running.

>:/usr/openv/netbackup/bin:$ ./bpps -x
NB Processes
------------
root 3938 72380 7 04:19 ? 00:00:02 /usr/openv/netbackup/bin/bpdbm
root 4518 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4519 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4572 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4574 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4582 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4588 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4592 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4593 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4594 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4608 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4609 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4610 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4611 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4843 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4871 72380 99 04:20 ? 00:00:02 /usr/openv/netbackup/bin/bpdbm
root 4910 72021 1 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4971 72021 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bprd
root 4997 72380 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bpdbm
root 5003 72032 0 04:20 ? 00:00:00 /usr/openv/netbackup/bin/bpcompatd
root 14651 72380 3 00:48 ? 00:06:28 /usr/openv/netbackup/bin/bpdbm
root 71732 1 0 Jul23 ? 00:22:02 /usr/openv/netbackup/bin/bpcd -standalone
root 72021 1 0 Jul23 ? 02:12:33 /usr/openv/netbackup/bin/bprd
root 72032 1 0 Jul23 ? 00:20:16 /usr/openv/netbackup/bin/bpcompatd
root 72193 1 0 Jul23 ? 00:03:41 /usr/openv/netbackup/bin/nbcssc -a NetBackup
root 72380 1 1 Jul23 ? 08:39:05 /usr/openv/netbackup/bin/bpdbm
root 72503 72380 4 Jul23 ? 1-00:56:41 /usr/openv/netbackup/bin/bpjobd
root 77476 1 0 Jul23 ? 00:00:18 /usr/openv/netbackup/bin/private/nbatd -c /usr/openv/var/global/vxss/eab/data
root 77525 1 0 Jul23 ? 01:05:32 /usr/openv/netbackup/bin/private/nbazd
root 77562 1 0 Jul23 ? 00:13:44 /usr/openv/netbackup/bin/nbaudit
root 78404 1 8 Jul23 ? 2-00:54:08 /usr/openv/netbackup/bin/nbdisco
root 78422 1 1 Jul23 ? 06:10:44 /usr/openv/netbackup/bin/nbevtmgr
root 78762 1 10 Jul23 ? 2-09:28:38 /usr/openv/netbackup/bin/nbemm
root 78779 1 7 Jul23 ? 1-20:59:05 /usr/openv/netbackup/bin/nbrb
root 78858 1 8 Jul23 ? 2-00:15:56 /usr/openv/netbackup/bin/nbjm
root 78886 1 1 Jul23 ? 11:06:16 /usr/openv/netbackup/bin/nbpem
root 78931 1 0 Jul23 ? 01:18:49 /usr/openv/netbackup/bin/nbstserv
oot 78988 1 0 Jul23 ? 00:07:13 /usr/openv/netbackup/bin/nbrmms
root 79088 1 1 Jul23 ? 09:42:35 /usr/openv/netbackup/bin/nbsl
root 79140 1 0 Jul23 ? 00:05:52 /usr/openv/netbackup/bin/nbim
root 79222 1 6 Jul23 ? 1-10:56:36 /usr/openv/netbackup/bin/nbars
root 79441 1 0 Jul23 ? 00:00:52 /usr/openv/netbackup/bin/nbvault
root 79482 1 0 Jul23 ? 00:35:44 /usr/openv/netbackup/bin/nbsvcmon
root 79660 72380 0 Jul23 ? 00:21:21 /usr/openv/netbackup/bin/bpdbm
root 79669 78858 0 Jul23 ? 01:00:52 /usr/openv/netbackup/bin/nbproxy dblib nbjm
MM Processes
------------
root 72011 1 0 Jul23 ? 00:01:27 vmd


Shared Veritas Processes
-------------------------
root 77374 1 1 Jul23 ? 11:19:54 /opt/VRTSpbx/bin/pbx_exchange

>:/usr/openv/netbackup/bin:$

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@Naveenyenigandl 

Not all master server processes are running and there seems to be excessive amount of bprd processes running.

The one that I'm missing immediately in your output is :
NB_dbsrv  (Adaptive Server Anywhere - VERITAS_NB - ASA Database service ).

Please stop NBU. 
Wait for processes to stop.
Check with bpps that everything has stopped.

Check disk space on the master - NBU will stop the relational database if disk gets full. This is to prevent database corruption. 

Check that bprd and bpdbm log folders exist under /usr/openv/netbackup/logs .
If not, create them.

Start NBU. 

Verify that all processes are running.
(See https://www.veritas.com/content/support/en_US/article.100002166.html )

If not, look for errors in these logs: bprd, bpdbm and /usr/openv/db/log/server.log.

Michal_Mikulik1
Moderator
Moderator
Partner    VIP    Accredited Certified

Hello,

other NBU commands (bppllist, bpstulist etc.) are running fine?

Login/Work with NBU Console is OK?

And was nbstlutil ever working?

Sometimes the errors desribed could be caused also by expired certificates/licenses, then also other NBU components would have issues.

regards

Michal

@Marianne , we will restart the services once and check.

@Michal_Mikulik1 , bppllist command is working, but i could not execute nbstlutil report command and the same was working till last week.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Please check  /usr/openv/db/log/server.log to see if there is a reason for the relational database going down.

@Marianne , i could see only the below details from server.log file.

 

I. 08/16 05:45:43. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 05:45
I. 08/16 05:50:18. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:50
I. 08/16 05:50:18. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:50
I. 08/16 05:56:36. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:56
I. 08/16 05:56:36. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:56
I. 08/16 06:00:44. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:00
I. 08/16 06:00:44. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:00
I. 08/16 06:05:44. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:05
I. 08/16 06:05:44. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:05
I. 08/16 06:08:07. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:08
I. 08/16 06:08:07. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:08
I. 08/16 06:11:02. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:11
I. 08/16 06:11:02. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:11
I. 08/16 06:19:39. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:19
I. 08/16 06:19:39. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:19
I. 08/16 06:23:38. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:23
I. 08/16 06:23:39. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:23
I. 08/16 06:25:45. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:25
I. 08/16 06:25:45. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:25
I. 08/16 06:29:08. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:29
I. 08/16 06:29:08. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:29
I. 08/16 06:33:08. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:33
I. 08/16 06:33:08. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:33
I. 08/16 06:40:16. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:40
I. 08/16 06:40:16. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:40
I. 08/16 06:45:46. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:45
I. 08/16 06:45:46. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:45
I. 08/16 06:46:14. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:46
I. 08/16 06:46:14. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:46
I. 08/16 06:52:38. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:52
I. 08/16 06:52:38. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:52
I. 08/16 07:00:54. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 07:00
I. 08/16 07:00:54. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 07:00
I. 08/16 07:05:47. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 07:05
I. 08/16 07:05:47. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 07:05

@Marianne , please find the below output of server.log.

 

I. 08/16 05:45:43. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 05:45
I. 08/16 05:50:18. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:50
I. 08/16 05:50:18. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:50
I. 08/16 05:56:36. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:56
I. 08/16 05:56:36. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 05:56
I. 08/16 06:00:44. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:00
I. 08/16 06:00:44. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:00
I. 08/16 06:05:44. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:05
I. 08/16 06:05:44. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:05
I. 08/16 06:08:07. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:08
I. 08/16 06:08:07. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:08
I. 08/16 06:11:02. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:11
I. 08/16 06:11:02. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:11
I. 08/16 06:19:39. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:19
I. 08/16 06:19:39. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:19
I. 08/16 06:23:38. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:23
I. 08/16 06:23:39. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:23
I. 08/16 06:25:45. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:25
I. 08/16 06:25:45. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:25
I. 08/16 06:29:08. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:29
I. 08/16 06:29:08. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:29
I. 08/16 06:33:08. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:33
I. 08/16 06:33:08. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:33
I. 08/16 06:40:16. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:40
I. 08/16 06:40:16. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:40
I. 08/16 06:45:46. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:45
I. 08/16 06:45:46. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 06:45
I. 08/16 06:46:14. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:46
I. 08/16 06:46:14. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:46
I. 08/16 06:52:38. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:52
I. 08/16 06:52:38. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 06:52
I. 08/16 07:00:54. Starting checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 07:00
I. 08/16 07:00:54. Finished checkpoint of "NBDB" (NBDB.db) at Fri Aug 16 2019 07:00
I. 08/16 07:05:47. Starting checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 07:05
I. 08/16 07:05:47. Finished checkpoint of "NBAZDB" (NBAZDB.db) at Fri Aug 16 2019 07:05

@Marianne , I could not see any error messages on server.log file. I have tried the same command"nbstlutil report" and am able to see the output now. This is really strange. 

sdo
Moderator
Moderator
Partner    VIP    Certified

1) Well, you definitely were missing the database process, so I wonder if that is running again now.  What does this command now show :

bpps -x | grep -i "dbs"

2) Did you restart NetBackup, or reboot the host ?

3) Did someone else restart NetBackup, or reboot the host ?

 

sdo
Moderator
Moderator
Partner    VIP    Certified

4) What do these commands show :

grep -v "^I" /usr/openv/db/log/server.log
grep -v "^I" /usr/openv/db/log/*.slg