About Retention in Storage Lifecycle Policies

[Eki_Rahmadian]

Hi Folks,

I'm new to this community, I'm hoping if anyone could help me with this case:

Currently in our environment, we have two netbackups master server in two different location, each server also have 1 media server installed and also attached with Quantum Disk in each site, all of them are installed under Windows 2008 R2 x64

Here's the setup,

DC1 : 
Master-Server-A
IBM Tape Library A, with 3 LTO 5 Tape Drives
Quantum VTL A

Media-Server-A
IBM Tape Library A, with 3 LTO 5 Tape Drives (Shared with Master Server)
Quantum OST Diskpool A

DC2 :
Master-Server-B
IBM Tape Library B, with 3 LTO 5 Tape Drives
Quantum VTL B

Media-Server-B
IBM Tape Library B, with 3 LTO 5 Tape Drives (Shared with Master Server)
Quantum OST Diskpool B

I configured two SLPs in two sites, like this:

- in DC1 I create SLP_A , first job is to backup to OST Disk with infinity retention, second job is replication to DC2 OST Disk with 1 year retention
- in DC2 I create SLP_B , first job is import with target retention, second job is duplicate to tape with retention infinity 

The Question is, when I checked the duplication result via restore job, it looks like the expired time is only 1 year, that means the retention is following the replication job in the first SLP, not infinity retention as I configured on the second SLP

What should I do so that the duplicated imported image will apply the infinity retention? 

Thanks in advance

Eki Rahmadian

[Yasuhisa_Ishika]

I suppose you have checked on DC2 master server.

By default, copy number in search criteria is set to 'Primary Copy', which means search result shows retention of first copy in DC2. Try to search for 'copy 2' - yoi will see copy with infinity retension is stored in some tapes.

[Eki_Rahmadian]

Thanks for the answer Yasuhisa, 

I've checked the copy 2 image, but it seems the image is still following the first SLP retention

And then we ran another test, change the retention into something like this:

- in DC1 I create SLP_A , first job is to backup to OST Disk with 1 week retention, second job is replication to DC2 OST Disk with 1 week retention
- in DC2 I create SLP_B , first job is import with target retention, second job is duplicate to tape with 2 weeks retention, and also using  override retention option in policy to 2 weeks retention

we got these results:

From my understanding, the override retention option is changing the tape media retention into two weeks, but when I check into the restore image, the image will expire in 1 week:

What am I missing here? what should I do so that the duplicated image will be expired after two weeks?

Thanks

Eki

[Marianne]

Please show us output of the following:

bpimagelist -client <client-name> -d 09/22/2014 -L

Please also double-check if retention levels may have been customised. On both sites, run this command and compare outputs:

bpretlevel -U

[Eki_Rahmadian]

Hi Marianne,

Here's the output:

bpimagelist

Client:            pwvxa6dbd201
Backup ID:         pwvxa6dbd201_1411377948
Policy:            TEST_OSTMED2_AIR
Policy Type:       MS-Windows (13)
Proxy Client:      (none specified)
Creator:           root
Name1:             (none specified)
Sched Label:       OSTMED2
Schedule Type:     FULL (0)
Retention Level:   1 week (0)
Backup Time:           9/22/2014 4:25:48 PM (1411377948)
Elapsed Time:      4 second(s)
Expiration Time:       9/29/2014 4:25:48 PM (1411982748)
Maximum Expiration Time:       10/6/2014 4:25:48 PM (1412587548)
Compressed:        no
Client Encrypted:  no
Kilobytes:         21684
Number of Files:   1
Number of Copies:  2
Number of Fragments:   4
Histogram:         0 0 0 0 0 0 0 0 0 0
DB Compressed:     no
Files File Name:   TEST_OSTMED2_AIR_1411377948_FULL.f
Previous Backup Files File Name:   (none specified)
Parent Backup Image File Name:   (none specified)
SW Version:        (none specified)
Options:           0x0
MPX:               0
TIR Info:          0
TIR Expiration:         1/1/1970 7:00:00 AM (0)
Keyword:           (none specified)
Ext Security Info: no
File Restore Raw:  no
Image Dump Level:  0
File System Only:  no
Object Descriptor: (none specified)
Previous BI Time:       1/1/1970 7:00:00 AM (0)
BI Full Time:           1/1/1970 7:00:00 AM (0)
Request Pid:       0
Backup Status:     0
Stream Number:     0
Backup Copy:       Standard (0)
Files File size:     1005
PFI type:     0
IMAGE_ATTRIBUTE:     0
Primary Copy:      2
Image Type:        2  (Imported)
Job ID:            34583
Num Resumes:       1
Resume Expiration:      1/1/1970 7:00:00 AM (0)
Data Classification:    (none specified)
Data_Classification_ID: (none specified)
Storage Lifecycle Policy:    OSTMED2_2_REPL
Storage Lifecycle Policy Version:    5
STL_Completed:      3
Remote Expiration Time:     9/29/2014 4:25:48 PM (1411982748)
Origin Master Server:  netbackup-dc2.ocbcnisp.com
Origin Master GUID:    4B0F07F1-E16C-45B2-9CC9-3C8C41193D3E
Snap Time:           1/1/1970 7:00:00 AM (0)
IR Enabled:      no
Client Character Set:     0
Image On Hold:     0
Indexing Status:   0
Copy number:       1
 Fragment:         TIR (-2147482648)
 Kilobytes:        1
 Remainder:        339
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaaD
 Host:             pwpmdsutd101
 Block Size:       1048576
 Offset:           0
 Media Date:            1/1/1970 7:00:00 AM (0)
 Dev Written On:   -1
 Flags:            0x100
 Media Descriptor:        1;Quantum;OSTMED22RPL_10.1.1.2;OSTMED22_REPL;OSTMED2_2
;0
 checkpoint:       0
 resume num:       0
 Key tag:          *NULL*
 STL tag:          *NULL*
 Copy on hold:     0
Copy number:       1
 Fragment:         TIR (-1)
 Kilobytes:        0
 Remainder:        1005
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaaD
 Host:             pwpmdsutd101
 Block Size:       262144
 Offset:           0
 Media Date:            1/1/1970 7:00:00 AM (0)
 Dev Written On:   -1
 Flags:            0x80
 Media Descriptor:        1;Quantum;OSTMED22RPL_10.1.1.2;OSTMED22_REPL;OSTMED2_2
;0
 checkpoint:       0
 resume num:       0
 Copy on hold:     0
Copy number:       1
 Fragment:         1
 Kilobytes:        21684
 Remainder:        0
 Media Type:       Disk (0)
 Density:          qscsi (0)
 File Num:         0
 ID:               @aaaaD
 Host:             pwpmdsutd101
 Block Size:       262144
 Offset:           0
 Media Date:            1/1/1970 7:00:00 AM (0)
 Dev Written On:   -1
 Flags:            0x0
 Media Descriptor:        1;Quantum;OSTMED22RPL_10.1.1.2;OSTMED22_REPL;OSTMED2_2
;0
 Expiration Time:      9/29/2014 4:25:48 PM (1411982748)
 MPX:              0
 retention_lvl:    1 week (0)
 Try to Keep Time:       1/1/1970 7:00:00 AM (0)
 Copy Creation Time:      9/22/2014 4:36:10 PM (1411378570)
 Data Format:      Undefined
 checkpoint:       0
 resume num:       0
 Copy on hold:     0
Copy number:       2
 Fragment:         1
 Kilobytes:        21684
 Remainder:        0
 Media Type:       NDMP (3)
 Density:          hcart2 (14)
 File Num:         1
 ID:               N10274
 Host:             pwpmdsutd101
 Block Size:       65536
 Offset:           2
 Media Date:           9/22/2014 5:05:08 PM (1411380308)
 Dev Written On:   12
 Flags:            0x0
 Media Descriptor:        ?
 Expiration Time:      10/6/2014 4:25:48 PM (1412587548)
 MPX:              0
 retention_lvl:    2 weeks (1)
 Try to Keep Time:      10/6/2014 4:25:48 PM (1412587548)
 Copy Creation Time:      9/22/2014 5:05:55 PM (1411380355)
 Data Format:      Tar
 checkpoint:       0
 resume num:       1
 Key tag:          *NULL*
 STL tag:          *NULL*
 Copy on hold:     0

bpretlevel -U

DC1

 Level  Days  Label
   0       7  1 week
   1      14  2 weeks
   2      21  3 weeks
   3      31  1 month
   4      62  2 months
   5      93  3 months
   6     186  6 months
   7     279  9 months
   8     365  1 year
   9   24855  infinity
  10       1  1 day
  11       6  167 hours
  12      20  20 days
  13      20  501 hours
  14   24855  infinity
  15   24855  infinity
  16   24855  infinity
  17   24855  infinity
  18   24855  infinity
  19   24855  infinity
  20   24855  infinity
  21   24855  infinity
  22   24855  infinity
  23   24855  infinity
  24   24855  infinity

DC2

 Level  Days  Label
   0       7  1 week
   1      14  2 weeks
   2      21  3 weeks
   3      31  1 month
   4      62  2 months
   5      93  3 months
   6     186  6 months
   7     279  9 months
   8     365  1 year
   9   24855  infinity
  10   24855  infinity
  11       6  167 hours
  12   24855  infinity
  13      20  502 hours
  14   24855  infinity
  15   24855  infinity
  16   24855  infinity
  17   24855  infinity
  18   24855  infinity
  19   24855  infinity
  20   24855  infinity
  21   24855  infinity
  22   24855  infinity
  23   24855  infinity
  24   24855  infinity

[Eki_Rahmadian]

The output of the command is attached

[Mark_Solutions]

Are you retention levels the same on both sites?

If site one for example has retention level 2 set to 1 week but site 2 has retention level 2 set to 1 month then even though your set it to 1 week on site 1 it will end up with 1 month on site 2 as it is the level that matters rather than the time .. so on each site check the master servers hosts properties - retention levels and make sure that they match.

It would be useful to see the output of the SLPs too .. nbstl -L

[Jeff_Foglietta]

I agree with Mark.. Let's see the output of nbstl -L...

Your SLP configurations are straight forward. At the source you have a retention period of 1 YR for the secondary operation (Replication). At the target you have retention set to "Target Retention" which will give your imported disk image a 1 YR retention. Your secondary operation at the target site (Dupe to tape) should be set with a "Fixed" retention of infinity. From your post it appears this is what you have done. Posting the output of the above command will help us see the SLP configuration clearer.

Just a recommendation here.. I'm going to assume your infinity retentions are reserved for quarterly/annual backups. That said, It is never a good idea to have infinity retentions residing on disk as you currently have at your primary site. Not only is it an expensive practice there is really no good business case to tie up disk storage "forever". I suppose you could make a business case for anything. I don't know how many restores you perform for images older than 1 yr, but I would think that number is low. You should duplicate all the infinity images to tape and reclaim that space by expiring the disk images. All long term retentions should always be written to tape. Anything longer than a year should be on tape only. If 1 yr is the minimum retention I would set the source backup to disk at a 1 yr retention, replication to target for 1 yr. Import at target for 1 yr (target retention) and dupe to tape "Fixed" for infinity. Should you need to restore after a years time you can do that from tape.

If your infinity retentions have a broader range than quarterly/annual backups you will need to keep an eye on your catalog growth... If your catalog isn't currently on a SAN disk which can be expanded as needed I would consider moving it there.

[Eki_Rahmadian]

Double Post

[Eki_Rahmadian]

Double Post

[Eki_Rahmadian]

I've created a ticket to Symantec Support, and got the answer So actually, the retention is exactly following the SLP that I configured, but when the restore GUI opened, the GUI shows the retention from the imported image, even when the copy 2 is selected as the primary copy, but once the imported image expired, the retention from the duplicated image (infinity) as i configured in the 2nd SLP is finally showed up. Is this practically a normal behaviour of the netbackup ?