01-09-2019 02:06 AM
Hi there,
We found a really old catalog backup in the Catalog which should long be expired.
When trying to expire it in the GUI it says Backup image expiration cannot be modified because its SLP processing is not yet complete (1573).
Doing this from commandline it says about the same (of course).
When trying to cancel its replication it gives also an error:
# nbstlutil cancel -backupid <Imagename>
Operation not successful: status = 220 (database system error)
According to Marianne in a previous issue 220 means 'no image found'.
The image is still active in a way:
# nbstlutil list -backupid <Imagename>
V7.6.0 I <Masterserverhostname> <Imagename> <Masterserverhostname> 1384237218 prd-cat-netbackup_catalog 7 0 zm_catalog_slp 3 false 1384242929 *NULL* 1 {00000000-0000-0000-0000-000000000000} 0 0 1384237225
V7.6.0 C <Masterserverhostname> <Imagename> 1 2147483647 1385360418 <stuname> 1 0 0 0 0 *NULL* 2147483647 0 2147483647 0 0 0 1 1
V7.6.0 F <Masterserverhostname> <Imagename> 1 1 0 @aaaa2 <Mediaserverhostname> *NULL* 0 6 1 3488240640 0 @aaaa2 *NULL* 1;DataDomain;<>DDhostname>;<diskpoolname>;<lsuname>;0
Now I am kind of stuck.
Solved! Go to Solution.
01-10-2019 07:58 AM - edited 01-10-2019 07:59 AM
We do root cause when a need for a SQL script comes up. Sometimes we're able to change the product so that either the situation won't arise, or NetBackup will be able to handle it.
With that in mind, I started searching our recent escalations for such a situation as yours. I found one that seems to match. (ET 3940173, Operation not successful: status = 220 (database system error))
Rather than a SQL script, this one gave a command option that worked for the customer. Try this:
>bpexpdate -backupid xxxxxxxxxxx -d 0 -force_not_complete
The report says that -force_not_complete allows expiration of incomplete SLP images.
BTW, status 220 doesn't always mean something not found - that's status 227. Often when a browse or bplist fails, NetBackup changes the 227 to 220.
01-09-2019 08:17 AM
Try running nbstlutil with the -force option:
nbstlutil cancel -backupid <Imagename> -force
Else contact support. Do you have SLP job in the activity console that ends with status code 1 ?
01-10-2019 04:00 AM
Hi Nicolai,
Thanks for the thinking.
# nbstlutil cancel -backupid <imagename> -force
option '-force' is no longer necessary and has been deprecated
Operation not successful: status = 220 (database system error)
We had some staus 1 in replication jobs because of a Data Domain file system issue, but that is with recent images.
The images which are stuck are from years ago, I can't figure out if there were any errors in those days.
01-10-2019 04:20 AM
You will need to log a call with Veritas Support.
They will need a database dump and then supply sql scripts to remove orphaned SLP entries.
01-10-2019 04:25 AM
01-10-2019 04:27 AM
Please let us know the outcome!
01-10-2019 05:51 AM - edited 01-10-2019 05:52 AM
I had a case like yours before. Most likely Veritas will provide you with a custom SQL job that cleans the EMM inconsistency.
01-10-2019 06:15 AM
I will post the results when (if) Veritas has solved this issue.
01-10-2019 07:58 AM - edited 01-10-2019 07:59 AM
We do root cause when a need for a SQL script comes up. Sometimes we're able to change the product so that either the situation won't arise, or NetBackup will be able to handle it.
With that in mind, I started searching our recent escalations for such a situation as yours. I found one that seems to match. (ET 3940173, Operation not successful: status = 220 (database system error))
Rather than a SQL script, this one gave a command option that worked for the customer. Try this:
>bpexpdate -backupid xxxxxxxxxxx -d 0 -force_not_complete
The report says that -force_not_complete allows expiration of incomplete SLP images.
BTW, status 220 doesn't always mean something not found - that's status 227. Often when a browse or bplist fails, NetBackup changes the 227 to 220.
01-14-2019 02:26 AM
Lowell Palacek provided the solution, thank you Lowell!
# bpexpdate -backupid <imagename> -d 0 -force_not_complete
Changing the expiration time of images that have not finished SLP processing can prevent completion of the SLP processing and lead to premature image expiration and potential data loss.
Are you SURE you want to proceed y/n (n)? y
Are you SURE you want to delete zmpbc003.rechtspraak.minjus.nl_1384237218 y/n (n)? y
Now the image has been removed. No commands can be issued against this image because it is gone:
# bpexpdate -backupid <imagename> -d 0 -force
no entity was found
# nbstlutil cancel -backupid <imagename> -force
option '-force' is no longer necessary and has been deprecated
No images or lifecycles matching criteria found.
We should have found this solution earlier because it is a documented feature of bpexpdate:
-force_not_complete
By default, an SLP-managed image or its copies cannot be expired if SLP processing is still in progress. The -force_not_complete option overrides this restriction and expires the image even if it is not SLP complete. Note that when you terminate further SLP processing of an image, other image copies may expire as well.
Thanks again Lowell.
02-02-2022 04:15 PM
Hi can I ask what is the impact to the SLP if a backup ends up with status 1, and that image is supposed to be duplicated? Thanks
02-02-2022 04:38 PM
Hi @jdelarosa
You should really start a new post rather than tacking onto an old (and solved) solution. Maybe a moderator can move this.
There is no impact to the SLP, it will perform the next step in the operation with the backup image that was saved during the first backup operation. Remember usually a status 1 implies one or more files could not be read cleanly during the backup impacting (potentially) thos files, but the rest of th ebackup is fine. The SLP will continue operations on what was backed up.
Cheers
David
02-02-2022 05:35 PM
Thank you! we are experiencing a somewhat related issue so I was browsing through some discussions. But I will create a new post like you mentioned. Thanks!
05-09-2023 08:37 AM
How to delete multiple id?
05-09-2023 03:57 PM
The bpexpdate command can take a list of backupids in a file as an argument (-Bidfile <file>).
Read the fine manual for more information
David
05-11-2023 06:13 PM
Hi @davidmoline
Please guide me specifically or maybe sent me the fine manual. Our storage server almost full because most of the data need to delete manually from console. I tried "bpexpdate -backupid xxxxxxxxxxx -d 0 -force_not_complete" before, but its time consuming. I have almost 200k database data entries that need to delete.
I appreciate any help that you can offer me. If you can help me, let me know.
AZMI
05-12-2023 12:14 AM
Have you tried using nbstlutil to cancel the SLP operations? This will give you the options available to you.
/usr/openv/netbackup/bin/admincmd/nbstlutil cancel -help