04-11-2024 05:55 AM
Hello Experts,
Oracle policy has been deleted and oracle client server has been decommissioned but still Net backup throwing error "the specified policy does not exist in the configuration database" with job policy "NULL" and Job schedule "NULL".
Can you please advise how to stop occurring this error.
04-11-2024 06:01 AM
You have to go to the client and check the crontab. i suppose that it is a linux/unix client.
To make your life easier go to /var/spool/cron/ and check every file in the directories. The files have usernames as names.
DO NOT EDIT THE FILES.
when you find how the backup is starting login as the user and use the crontab -e command to delete or commend the line(s) that start the backup
04-11-2024 06:04 AM
Hello StefanosM,
But the all the client servers are decommissioned which are throwing these errors.
04-11-2024 06:08 AM
are you sure?
check again. ping the decommissioned servers.
Another check you can do is to check the crontab of the master server.
In theory every system that is in the servers list of the master server can initiate the backup
04-11-2024 06:16 AM
I am with @StefanosM , something is likely calling a oracle related backup/restore/archive. Tracking down the client can be a bit difficult.
My recommendation is to use vxlogview for a short time frame around the time where error shows up to capture the host.
e.g Show logs for 2 min between specific time frames
# vxlogview -p 51216 -b "07/21/2023 1:30:00 PM" -e "07/21/2023 1:32:00"
or everything that happened in the last 5 min.
# vxlogview -p 51216 -t 00:05:00
Alternative, create the policy and schedule again, then the name should show up in the GUI
04-11-2024 06:16 AM
Hello,
Yes, i am sure .. client server is decommissioned and i am not able to ping the decommissioned server.
04-11-2024 06:36 AM
Logs generated...
[root@<Master server> jobs]# vxlogview -p 51216 -t 00:11:00 | grep -i <client name>
04/11/2024 15:22:03.198 [Info] V-116-82 received start user backup request for client=<client name>, policy=?, schedule=?, stream #=0, client type=Oracle (4), parent jobid=-1
04/11/2024 15:22:03.199 [BPCRConnFactory::requestBPCRConnection] inserting connection request (back) BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440f9850)(BPCRConnFactory.cpp:1041)
04/11/2024 15:22:03.199 [BPCRConnFactory::requestBPCRConnection] inserting connection request (back) BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440c5970)(BPCRConnFactory.cpp:1041)
04/11/2024 15:22:03.199 [BPCRConnection::linkToRequester] Connection BPCRConnection(state=1, hadError=0, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle) requester=0x7f12440f9850(BPCRConnFactory.cpp:210)
04/11/2024 15:22:03.199 [BPCRConnFactory::handle_input (evaluation)] connection request ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440c5970)) not granted because one ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440f9850, connection= BPCRConnection(state=1, hadError=0, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle))) is already active in a serialized pool(BPCRConnFactory.cpp:1441)
04/11/2024 15:22:03.200 [BPCRConnFactory::handle_input (evaluation)] connection request ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440c5970)) not granted because one ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440f9850, connection= BPCRConnection(state=2, hadError=0, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle))) is already active in a serialized pool(BPCRConnFactory.cpp:1441)
04/11/2024 15:22:03.200 [Info] V-116-14 CLIENT <client name> POLICY *NULL* SCHED *NULL* EXIT STATUS 230 (the specified policy does not exist in the configuration database) VBRF 1 230
04/11/2024 15:22:03.202 [BPCRConnFactory::handle_input (evaluation)] connection request ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440c5970)) not granted because one ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440f9850, connection= BPCRConnection(state=2, hadError=0, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle))) is already active in a serialized pool(BPCRConnFactory.cpp:1441)
04/11/2024 15:22:03.202 [BPCRConnFactory::handle_input (evaluation)] connection request ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440c5970)) not granted because one ( BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440f9850, connection= BPCRConnection(state=2, hadError=0, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle))) is already active in a serialized pool(BPCRConnFactory.cpp:1441)
04/11/2024 15:22:03.214 [BPCRConnFactory::returnConnection] connection returned was found on the active list BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440f9850, connection= BPCRConnection(state=3, hadError=1, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle))(BPCRConnFactory.cpp:1115)
04/11/2024 15:22:03.214 [BPCRConnectionRequest::unlinkFromRequester()] connection not reusable, will close BPCRConnection(state=4, hadError=1, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle)(BPCRConnFactory.cpp:1713)
04/11/2024 15:22:03.214 [BPCRConnection::linkToRequester] Connection BPCRConnection(state=1, hadError=0, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle) requester=0x7f12440c5970(BPCRConnFactory.cpp:210)
04/11/2024 15:22:03.214 [Warning] V-116-121 cannot send mail to <client name>
04/11/2024 15:22:03.217 [BPCRConnFactory::returnConnection] connection returned was found on the active list BPCRConnectionRequest(pool=6, connectionName=<client name>::BPCD::oracle, requester=0x7f12440c5970, connection= BPCRConnection(state=3, hadError=1, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle))(BPCRConnFactory.cpp:1115)
04/11/2024 15:22:03.217 [BPCRConnectionRequest::unlinkFromRequester()] connection not reusable, will close BPCRConnection(state=4, hadError=1, activeFDcnt=3, connectName=<client name>::BPCD::oracle, hostName=<client name>, userName=oracle)(BPCRConnFactory.cpp:1713)
04-11-2024 06:43 AM
Hello Nicolai,
any more logs required? please let me know.
Note: Client server was decommissioned in February 2024.
04-11-2024 07:33 AM - edited 04-11-2024 07:34 AM
this is find against job id in vxlogview
[root@ tmp]# cat vxlog.txt | grep -i 107722531
04/11/2024 16:22:04.080 [JobManager_i::doForgottenJobCleanup] (0x22ba8c0) job has been forgotten, perform cleanup, jobid=107722531(JobManager_i.cpp:2607)
04/11/2024 16:22:04.080 [JsnParamFile::init] m_filePathExists=0 m_filepath=/usr/openv/netbackup/db/jobs/restart/107722531.j m_fileAltPathExists=0 m_altfilepath=/usr/openv/netbackup/db/jobs/restart_alt/107722531.j isBasicBackup=0 isStdBackup=0 isCommandJob=0(JsnParamFile.cpp:256)
04/11/2024 16:22:04.080 [JsnParamFile::ReadParamFile] json_load_file failed, param file /usr/openv/netbackup/db/jobs/restart_alt/107722531.j json_error_t line -1 pos: 0 col: -1 text: unable to open /usr/openv/netbackup/db/jobs/restart_alt/107722531.j: No such file or directory src: /usr/openv/netbackup/db/jobs/restart_alt/107722531.j(JsnParamFile.cpp:360)
04/11/2024 16:22:04.080 [JsnParamFile::ReadParamFile] status 821 param file /usr/openv/netbackup/db/jobs/restart_alt/107722531.j(JsnParamFile.cpp:445)
04/11/2024 16:22:04.080 V-117-227 [JobManager_i::doForgottenJobCleanup] Cannot read param file for jobid=107722531
04-11-2024 07:43 AM - edited 04-12-2024 04:54 AM
Attached log file ( Vxlogview of last 5 mins) and job id is
107722531 | Backup | Done | 230 | *NULL* | *NULL* | client name | Apr 11, 2024 4:21:58 PM | 00:00:00 | Apr 11, 2024 4:21:58 PM | 100% | oracle | 107722531 | master server |
04-12-2024 12:42 AM - edited 04-12-2024 12:49 AM
I was not able to find a host name for job id 107722531 in the attached log.
You may have to bump the debug level.
Alternative use wireshark / tcpdump and listen for traffic on port 1556/tcp. You should be able to see what host initiate the backup request.
If you have a jobid , you can query logs with:
vxlogview -p 51216 -X "jobid=107722531"
As mentioned earlier, it's a tricky task.
04-12-2024 12:47 AM
I suggest to create the same policy/schedule/client and see if the backup will run or not.
04-12-2024 02:41 AM
Hello @Kak4_12 ,
inspect Master Server crontab (or any other scheduler if used) for a command "bpbackup -i -p <deleted policy>". Theoretically such a script could also be hosted on any server specified on Host properties\Master\Servers\Additional Servers.
No need to upload more numerous logs here.
Regards
Michal
04-15-2024 12:09 AM
Or use wireshark / tcpdump to inspect what hosts initiate traffic at port 1556 , 13724 or 13782.