Media Expire

[H_Sharma]

Hello Experts,

We found in our environment media does not expire automatically and we have to expire media manually Please help.

Environment is as below.

Master :- win 2008 R2 Cluster 7.6.0.3

Media :- win 2008 r2 7.6.0.3

[RiaanBadenhorst]

Some background information or evidence you collected when determining this would be required.

[Marianne]

Please show us what you see.

What do you see in 'Tape Summary' report with Verbose option selected?

Show us evidence of one tape that has passed expiration date:
nbemmcmd -listmedia -mediaid <media-id>

Can you see Image Cleanup job running every 12 hours?
Are you aware that tapes are only recycled during Image Cleanup Job?

Have you checked if NoExpire (in <install path>\Netbackup\bin\) was perhaps created on the currect active master server node at some point?

[Nicolai]

Do you mix retension on tape ?

In that case you will have to wait until ALL images has been expired before tape is re-used.

https://www.veritas.com/community/blogs/understanding-how-netbackup-writes-tape

[mph999]

The ouput of bpmedialist -m <media ID> for a problem media would be useful. before it is manually expired.

Thanks,

Martin

[Marianne]

Don't you just love this. A statement is made accusing NBU of not working as designed. We ask for evidence, we suggest a couple of things to look at. And then.... silence. No response over the next 8 hours. Please help us to help you?

[H_Sharma]

Hi Marianne,

What do you see in 'Tape Summary' report with Verbose option selected? It seems fine

I will surely provide you details of nbemmcmd -listmedia -mediaid <media-id> after some time as of now we have expired all the old tapes.

Can you see Image Cleanup job running every 12 hours? Yes its running after every 12 hours Every time it gives us below

25/01/2016 15:21:08 - Info bpdbm(pid=3912) image catalog cleanup         
25/01/2016 15:21:08 - Info bpdbm(pid=3912) Cleaning up tables in the relational database     
25/01/2016 15:21:08 - Info bpdbm(pid=3912) deleting images which expire before Mon Jan 25 15:21:08 2016 (1453731668) 
25/01/2016 15:21:09 - Info bpdbm(pid=3912) deleted 0 expired records, compressed 0, tir removed 0, deleted 0 expired copies
the requested operation was successfully completed(0)

Yes i checked NoExpire (in <install path>\Netbackup\bin\ but didnot find this file.

[Marianne]

What is the soonest expiration date/time you see in verbose 'Tape Summary' report?

Can you check Image Cleanup job again the 1st time it runs after the tape expirarion has passed?

[H_Sharma]

Please find the details of one of the media thats already expired.

C:\Program Files\Veritas\NetBackup\bin\admincmd>bpmedialist -m 00056
Server Group = UNRESTRICTED_SHARING_GROUP

 id     rl  images   allocated        last updated      density  kbytes restores

           vimages   expiration       last read         <------- STATUS ------->

           On Hold
--------------------------------------------------------------------------------

00056  11    333   20/12/2015 00:00  30/12/2015 11:27   hcart  3484749828     0

              MPX   27/01/2016 11:27        N/A         EXPIRED FULL
           0

[Marianne]

Did you see that top line in bpmedialist output? " On Hold " Why is that? Seems you have 'Search and Hold' option in your environment? With the hold option your backups don't get expired.

[Nicolai]

"on hold" field is 0 - I think that mean its not on legal hold.

Media 00056 also show as "EXPIRED FULL". The media cleaning run every 12 hours in Netbackup, meaning the tape worst case could be available by 27/1/2016 11:27 + 12 hours.

[Marianne]

Thanks for that Nicolai. I could not see the '0'  on the mobile version of Connect.

We need to see what is happening in the 1st Image Cleanup after 27/01/2016 11:27.

If still not unassigned, you will need bpdbm log on the master server.

The last time that non-expiration of tapes was caused by a bug, was some years ago in NBU 7.0.1.
The bug was described as follows:

The bpexpdate -deassignempty command is not being run automatically by the scheduler, and empty tapes, which hold no valid images, are not being made available for backups 

There was an EEB at the time and has not been seen in newer version.

[Suchet_Siddu_J_]

Hi

Could you let me know, the existence of  "NOExpire" file under "<install path>\Netbackup\bin\.

If the file exists delete that file.

[mph999]

If the NOexpire file exists DO NOT DELETE THE FILE.

First, find out why it is there, for example has a colleague created it to prevent some backups expiring, perhaps for legal reasons ???  Only when you know why it was created should it be removed else otherwise important images could potentially be lost.

(As a side note, the Noexpire  file should not be used for 'holding ' images, it is 'unsupported' as in should ONLY be used by tech support as its use  can lead to further issues in an environment).

[mph999]

Can you look in the  bptm log :

The command bptm -delete_all_expired should be running every 10 minues that should release expired media.  This is an additional cleanup command to bpexpdate -deassignempty command that was correctly mentioned by Marianne.

For example, from my system:

root@womble bptm $ grep delete_expired *

07:54:49.635 [19439] <4> delete_expired_media: (-) Translating EMM_ERROR_Success(0) to 0 in the NetBackup context
07:55:02.519 [19440] <4> delete_expired_media: (-) Translating EMM_ERROR_Success(0) to 0 in the NetBackup context
 

We see the command should be frequently.  This is NOT seen in Activity Monitor.

Here is my expired media :

--------------------------------------------------------------------------------
A00001   0      0   01/27/2016 11:14  01/27/2016 11:14   hcart           0     0
                0   01/29/2016 08:02        N/A         EXPIRED
 

Hopefully the command should run at about shortly .....

< a few minutes later>

07:55:02.519 [19440] <4> delete_expired_media: (-) Translating EMM_ERROR_Success(0) to 0 in the NetBackup context
08:14:50.193 [20767] <4> delete_expired_media: (-) Translating EMM_ERROR_Success(0) to 0 in the NetBackup context
08:34:49.604 [21617] <4> delete_expired_media: (-) Translating EMM_ERROR_Success(0) to 0 in the NetBackup context

... and the tape then does not appear in bpmedialist

root@womble bptm $ bpmedialist -m A00001
requested media id is not assigned to this host in the EMM database

So first, I would check if you can see similar lines in the bptm log - I'm not sure if the NOexpire flag would affect this but no harm in checking to see if it is there.

[Marianne]

And another 5 days go by without any response from H.Sharma.

Do we assume that you do not have a real issue?
And that tapes are indeed recycled when Image Cleanup runs?

[H_Sharma]

Hi All,

Sorry for delayed response.

1:- We dont mix retentions.

2:- There is no file Noexpire in media servers or master as well.

3:- I checked the BPTM log on media servers and could not find delete_expired_media command. May be as of now we dont have any tapes which got expired. Not sure why delete_expired_media its not there.

Please let me know if anthing is required.

[Marianne]

You never replied to Nicolai's post on 28 Jan: ' Media 00056 also show as "EXPIRED FULL". The media cleaning run every 12 hours in Netbackup, meaning the tape worst case could be available by 27/1/2016 11:27 + 12 hours.' Did you ever check status of this (and other tapes) 12 hours after expiration?

[H_Sharma]

Yes marianne,

I have checked and media does not get expired on its own even after 1-2 days.

[Marianne]

We've been trying to help you since 26 Jan, but response from you has been few and far between.

When you eventually reply, you give no useful info that will help us to help you.

Same with your 'Drive Down' issue that you post over and over...

Why are going though the trouble of posting when you have no intention of providing useful feedback?

I wish you all the best.....