cancel
Showing results for 
Search instead for 
Did you mean: 

How to delete orphend policy from Netbackup

Baski
Level 5
Partner Certified

Hi Team,

 

We are facing one strange issue. Some one create a policy with xxxxxx/ ( with slash) and this  backup is failing with EC-84, but Original backup is running fine without any issue.  We have deleted the policy from GUI, but still it is running. We have checked in db/cass/ folder there we find the / policy name. We have deleted the policy there also. But still it is running. No idea, where it is running.

 

We have updated/run the below command, but no use. Stil the job is running.

 

nbpemreq -updatepolicies

 

is any help how to delete the policy from the NBU without recycle the NBU services ?

 

Thanks for your help.

1 ACCEPTED SOLUTION

Accepted Solutions

Jeff_Foglietta
Level 5
Partner Accredited Certified

Since the policy has been deleted and continues to run for whatever the reason, first thing you should do is rename the script the job runs on the Oracle client. This will prevent the job from running albeit with a failed status code attached to it but at least you stop it from running. At that point I would certainly look to restart the services for NetBackup WITHOUT rebooting your production servers. Make sure you suspend or cancel jobs currently running. Make note of the cancelled jobs and restart them once the services come back up. Resume any suspended jobs.

Questions...

Snipped from one of your posts to Marianne

"I can see the original policy only, but not the "\" policy, which we already removed from the NBU policy GUI." ( I think you meant "/" in this sentence..yes?)

What is this original policy? Does it have the same name sans the /??????????

Also, out of curiosity have you attempted to run the find command on the policy name? I ask this because I don't know how a policy was able to be created with a "/" in the string to begin with and I'm curious to see if there may now be a directory out there with the name DB-ORA-DB_SID-ARCH-NET. Just for kicks and giggles run the following

# find / -name DB-ORA-*

or

# find / -name DB-ORA-* -print (If Solaris)

 

View solution in original post

15 REPLIES 15

Mark_Solutions
Level 6
Partner Accredited Certified

You could try a bppldelete policyname but i suspect that it is just embeded in the scheduler.

Last time i saw one like this i had to recreate the policy with the same name .. add a schedule with the same name but with no window - run the nbpemreq command again to clear it out of the scheduled timers list and then delete the policy.

See if that works for you

 

Luiz_Prina
Level 4
Employee

Do you if there's a script launching the backup from its client? 

We faced that a time ago and we need to check the client's crontab.

 

SymTerry
Level 6
Employee Accredited

By still running do you mean that the policy is still listed or are you refering to an active job from that policy? 

If an active job, Get the job numbers by running /usr/openv/netbackup/bin/admincmd/bpdbjobs -report

Then kill those jobs with by running <install path>/bin/admincmd/bpdbjobs -delete <job number [job number,job number,...]>

Leed_Engineer
Level 4
Partner Accredited Certified

Hi ,

    Try to find the lock file under /usr/openv/netbackup/bin which should be cleanup.lock then Check which process is using it using fuser command in UNIX/LINUX.

 

Check if you can kill the process that hold that file then move the file to somewhere else like  cleanup.lock.orig then try to delete the policy again.

 

Baski
Level 5
Partner Certified

Hi All,

 

Thank you for comment here,

There are no jobs in crontab.

It is very difficult to cancel the jobs. Since it is archive backup ( it will run every hour)

Unable to delete the policy, below is the error.

 

 sudo ./bppldelete DB-ORA-DB_SID-ARCH-NET/
failed accessing daemon lock file

Please help here

 

Baski
Level 5
Partner Certified

Hi Leed,

Thanks for your information,  I tried below, but no luck.

 

 ls -ltr cleanup.loc*
-rw-------   1 root     root          17 Apr 23  2013 cleanup.lock
-rw-------   1 root     root          17 Sep 26 12:44 cleanup.lock_1
-rw-------   1 root     root          17 Sep 26 12:44 cleanup.lock_2
-rw-------   1 root     root          17 Sep 26 12:45 cleanup.lock_3
-rw-------   1 root     root          17 Sep 26 13:39 cleanup.lock_4

 

 $  fuser cleanup.lock_1
cleanup.lock_1:
 $ fuser cleanup.lock_2
cleanup.lock_2:
  $ sudo fuser cleanup.lock_2
 cleanup.lock_2:
  $ sudo fuser cleanup.lock_3
cleanup.lock_3:
  $ sudo fuser cleanup.lock_4
cleanup.lock_4:

  $ sudo fuser cleanup.lock
cleanup.lock:

sudo ./bppldelete DB-ORA-DB_SID-ARCH-NET/
failed accessing daemon lock file

 

But I can found below lock files .

 ls -ltr *.lock*
-rw-------   1 root     root          17 Oct 30  2010 sync.lock
-rw-------   1 root     root          17 Apr 23  2013 cleanup.lock
-rw-------   1 root     root          16 Jul 10 12:34 bpcompatd.lock
-rw-------   1 root     root          16 Jul 10 12:34 bpdbm.lock
-rw-------   1 root     root          16 Jul 10 12:34 bpjobd.lock
-rw-------   1 root     root          16 Jul 10 12:34 bprd.lock
-rw-------   1 root     root          17 Sep 26 12:44 cleanup.lock_1
-rw-------   1 root     root          17 Sep 26 12:44 cleanup.lock_2
-rw-------   1 root     root          17 Sep 26 12:45 cleanup.lock_3
-rw-------   1 root     root          17 Sep 26 13:39 cleanup.lock_4
-rw-------   1 root     root          17 Sep 26 15:54 catbackup.lock

 

 

Please let me know, which file I can check.

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Try to delete the policy by adding a \ before the / :

sudo ./bppldelete DB-ORA-DB_SID-ARCH-NET\/

(there is a \ at the end followed by / - not a V ! )

Baski
Level 5
Partner Certified

HI Marianne,

 

Thank you for comment here. I tried wheat you said as above, but still no luck. Same error again.

 

 

/usr/openv/netbackup/bin/admincmd/bppldelete DB-ORA-DB_SID-ARCH-NET\/ ( This is not v  \ and /)

failed accessing daemon lock file

I think this is Ghost policy in our environmet.

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Can you see this name in class folder?

cd /usr/openv/netbackup/db/class

ls -l DB-ORA-DB_SID-ARCH*

If so, we will be able to get rid of it at OS level.

Baski
Level 5
Partner Certified

Hi Marianne,

 

I can see the original policy only, but not the "\" policy, which we already removed from the NBU policy GUI. But no idea, where this orphend policy starting and backups are keep on failing. I spoke to some one from  my company  ( senior ), he said" reboot(restart the services) to solve the issue.

This is production server, we need take lot of approvals before reboot/restart services.  If there is no chance,  then I will raise change to reboot the master server.

 

Thanks

sksujeet
Level 6
Partner Accredited Certified

Can you please output of bppllist DB-ORA-DB_SID-ARCH-NET/ -L

How did you create this policy at the first place, / is not allowed in policy and I am wondering how you were able to create this policy.

It looks like a oracle policy and seems to be getting trigged via some kind of script

Baski
Level 5
Partner Certified

Hi Sazz,

 

I have told in my previous updates. some one created it and we have deleted the policy from NBU GUI. But no idea where now the job is running now. there is no class DB also.

Thanks,

Jeff_Foglietta
Level 5
Partner Accredited Certified

Since the policy has been deleted and continues to run for whatever the reason, first thing you should do is rename the script the job runs on the Oracle client. This will prevent the job from running albeit with a failed status code attached to it but at least you stop it from running. At that point I would certainly look to restart the services for NetBackup WITHOUT rebooting your production servers. Make sure you suspend or cancel jobs currently running. Make note of the cancelled jobs and restart them once the services come back up. Resume any suspended jobs.

Questions...

Snipped from one of your posts to Marianne

"I can see the original policy only, but not the "\" policy, which we already removed from the NBU policy GUI." ( I think you meant "/" in this sentence..yes?)

What is this original policy? Does it have the same name sans the /??????????

Also, out of curiosity have you attempted to run the find command on the policy name? I ask this because I don't know how a policy was able to be created with a "/" in the string to begin with and I'm curious to see if there may now be a directory out there with the name DB-ORA-DB_SID-ARCH-NET. Just for kicks and giggles run the following

# find / -name DB-ORA-*

or

# find / -name DB-ORA-* -print (If Solaris)

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Do you have bprd log folder on the master? You will be able to see in the bprd log where the policy is requested from.

KDob
Level 3
Partner Accredited

http://www.symantec.com/business/support/index?page=content&id=TECH43177

http://www.symantec.com/connect/forums/unknown-job-stuck-activity-monitor

I know these are for jobs in the activity monitor, but if you also delete the temporary policy execution manager file in Install Location/netbackup/db/jobs (exact name escapes me at the moment) that will force it to be rebuilt upon the restarting of all the services.

If you truely were able to delete the policy, it should not reappear in the new work list created by nbpemreq.

Note:  If the policy was an Oracle Policy which called RMAN scripts in order to execute, once launched the script will continue to run; whether or not there is a policy physically present in the Policy List on the Master.

Hope this helps.....kind of round about way of getting to where you want to be; but, only solves the problem if the problem was your policy was still being executed by PEM.

I have been no help at all, I realize, if you see the Policy in your Policy List and just want to delete it.

To delete the policy, drop all NBU services, go to Install Location/netbackup/db/class and simply delete the directory named: DB-ORA-DB_SID-ARCH-NET\/

#rm -fR "Install Location"/netbackup/db/class/policynametodelete   (as Marianne suggested above)

Let us know if you solved your issue and what the final fix was....Thanks!