03-21-2019 11:42 AM
I have seveal tapes that have various retentions levels. Like 60 days or 1 year etc... All of the last written dates have come and gone, when look i see Data expiration /Expiration Date/Time is set to 2038 on all of them, so tapes are not expiring. I have checked all the policys and SLP's and none have this date. I have checked several by runing report "Images on tape" none have retention set to "infinity" only the set time within the Policy/slp like 1 year.
My question is what may have caused this to happen, set retention/expiration date/time to 2038?
03-21-2019 12:50 PM
03-21-2019 01:00 PM
2038 is the year NetBackup sets when it is setting to infinity. So somehow your tapes were set to infinity. It is possible to change the retention values on live systems, so even though the retention value, like 09 or 10 may now be set to 1 year, it is possible to have been infinity when the jobs ran.
Some SLP can be defined to not expire copy 1 until copy 2 is complete, and may set them as infinity in the interim. If you can run a check of your slp and make sure they are all complete that will eliminate that as a possibility.
I have seen it in the past, that if you set ONE image on a tape into the future, the tape expiration (not the other images) gets pushed to that date. Even though you change the image back, sometimes the tape expiration remains in the future.
Please run these commands taking NO action to an output file and VERIFY the images are the ones you want to modify.
Expiring the wrong batch of images can be a resume generating event!
You should be able to run a couple of commands:
List images on these tapes.
For each image, recalculate the retention and expiration for 1 year - that should make it all better.
If that does not work, then
for each tape - set them to expire tomorrow, then run the recalulation and expiration to 1 year.
03-22-2019 12:27 AM
Could you please show us what you see for one of the tapes with Infinity data expiration?
Output of 'nbemmcmd -listmedia -mediaid <media-id>'
Output of 'bpimmedia -U -mediaid <
One 'feature' of SLPs is that expiration will initially be set to Infinity until ALL secondary operations have been completed.
Only when all copies of duplications and/or replications have successfully completed will the SLP retentions be applied.
So, it might be possible that there are are outstanding secondary operations against some images.
03-22-2019 12:58 AM - edited 03-22-2019 12:59 AM
Agree with Marianne
Netbackup set infinity retention on all SLP images not processed, once SLP operation is carried out, the original retension is applied.
Behavior is documnted here:
Storage Lifecycle Policy (SLP) backup images have original backup copy expiration date set to infinity
03-22-2019 02:24 AM
Sorry to spoil the party ....
Previously, NBU most certainly did set the copy 1 of an SLP to infinity, prior to additional copies being completed.
Not anymore, it changed 'a while back' .... Now, copy 1 will be set to the longest expire time of any of the copies, and then will revert to 'it's own expire time' after the copies are made.
Eg. I just ran an SLP, copy 1 has a retention of 1 hour, copy 2 was a week.
Looking in NBDB Copy table before the duplication was made, at copy 1 is set to 1 week, not infnity.
1: "ImageCopyKey" unsigned bigint = '1931'
2: "ImageKey" unsigned bigint = '1945'
3: "CopyNumber" integer = '1'
6: "ExpireTime" bigint = '1553850914'
root@gpk630r2f-18 db $ bpdbm -ctime 1553850914
1553850914 = Fri Mar 29 09:15:14 2019
M
03-22-2019 02:41 AM
Good feedback - Thanks
03-22-2019 02:58 AM
Can't remember when it changed, 7.7 something I think.
I guess the thought was that there is no point keeping copy 1 any longer than the time that 'every' copy of the image would have been expired by.
03-22-2019 06:30 AM
Hi @mph999
Thanks for this.
It would just be 'nice' if documentation could be updated accordingly.
Herewith extract from 8.1.2 Admin Guide I pdf :
About ensuring successful copies using lifecycles
The process to create copies as part of a storage lifecycle policy differs from the
process to create copies as set up in a policy. The policy’s Configure Multiple
Copies dialog box includes the option to Fail all copies. That option means that
if one copy fails, the remaining copies can be set to either continue or fail.
In an SLP, all copies must be completed. An SLP initially tries three times to create
a copy. If no copy is created, NetBackup continues to try, but less frequently.
The successful completion of copies is important because an SLP does not allow
a copy to be expired before all copy operations in the SLP are complete. NetBackup
changes the retention period of a copy to Infinity until all copies are created. After
all copies are complete, the retention returns to the level as set in the policy.
Online (8.1) version of Admin Guide 1:
https://www.veritas.com/support/en_US/doc/18716246-126559472-0/v42386889-126559472
The successful completion of copies is important because an SLP does not allow a copy to be expired before all copy operations in the SLP are complete. NetBackup changes the retention period of a copy to Infinity until all copies are created. After all copies are complete, the retention returns to the level as set in the policy.
03-22-2019 11:14 AM
Thanks for the responses, here is a .txt of the output from >nbemmcmd -listmedia -mediaid 100718
reading some of the other posts and yes this policy is using an SLP that includes a from Data Domain to Tape for 1 year retention.
I don't see in the output inlist media where the copy failed ? All so is there a way to fix this, or do i have to go through and bpexpdate each one?
03-22-2019 03:47 PM
... I'l see if I can track down the author of the manual.
I think my test shows for certain it has changed ... i always doubt myself with things like this.
03-22-2019 03:49 PM
Unfortunatley, you will have to change the retention times manually. Persoanlly, I would just script it.
03-24-2019 12:07 AM - edited 03-25-2019 04:13 AM
So, it seems that something is wrong here.
nbemmcmd shows retention as 8 which is supposed to be 1 year, but with INFINITY expiration.
Can you list images on this media (there are only 5) and then show us output for one of the images?
bpimagelist -backupid <backup_id> -L
**** EDIT ***
Looking again at media output, there is another possibility.
Data Expiration: INFINITY
Last Written: 09/27/2016 05:17
....
Images: 5
Valid Images: 5
Retention Period: 8
Maybe someone in your company has manually changed expiration date to INFINITY.
This could've been done any time after Last Written date (2016).
Changing expiration with bpexpdate will change the Data Expiration date, but not Retention Period (level).
03-25-2019 09:24 AM
You wouldn't happen to have one(Script) i could use? looking deeper into this i have over 100 that i will have to check. I do have some data that is condidered forever data, and will have to mindfull not to clear any of these tapes.
Thanks for responses.
03-26-2019 01:02 AM
Not for Windows no, do you have a Linux box - I would copy a list of images to Linux, and use a script to create commands, then copy/ paste back to Windows.
cat images.txt |while read LINE
do
echo "bpexpdate -d <newdate> -force -backupid $LINE" >myscript.sh
done
03-26-2019 08:03 AM
03-26-2019 08:12 AM
do you separate your tapes by retention in specific volume pools?
Even if not, you can use filtering to filter only tapes with retention X
Select all the tapes, ctrl-C - paste in an excell spreadsheet - delete all but the media ID column.
Paste those media id into a txt file. Say media.txt
for TAPE in `cat media.txt`
do
echo $TAPE
for IMAGE in `bpimmedia -l -mediaid $TAPE | egrep -v NULL | awk '{print $4}'`
do
echo $IMAGE
/usr/openv/netbackup/bin/admincmd/bpexpdate -recalculate -backupid $IMAGE -ret ## -force
done
done
NOTE - you can add -copy # in recalc command if you need to not recalc your first copy on data domain!
03-27-2019 01:23 AM
@psbrizzi wrote:
.... looking deeper into this i have over 100 that i will have to check. I do have some data that is condidered forever data, and will have to mindfull not to clear any of these tapes.
That sounds quite dangerous....
Best to have a good look at different retention requirements for different types of data and create new SLPs to be used in the relevant policies.