Can someone explain this SLP behavior, I don't get it

[Volker_Spies1]

Hi all,

I have the following goal:

We have a large Fileserver, wich needs a full backup at weekends. This backup has a 6 week retention. But once a moth I need a backup with a retention of 60 weeks. So i changed the storage that the "month" schedule uses to a SLP. This SLP does the backup with 6 weeks to the week pool and than a duplicate with 60 Weeks retention.

But when I have a look at the incomplete SLP copies right now, it seems that now ALL backups from that server, even ones taken 5-6 weeks ago, will be duplicated to 60 weeks?

The server has ~6TB Storage and when I have a look in nbstluitl report, NBU tries to duplicate 41TB right now and that number is growing.

Can some one explain this behavior?

As I said, what I want to archieve is, that once a month a 6 weeks retention backups is duplicated to 60 weeks, and we can remove it from the library.

Volker

[sksujeet]

So i changed the storage that the "month" schedule uses to a SLP. This SLP does the backup with 6 weeks to the week pool and than a duplicate with 60 Weeks retention.

Not the right way in that case for this particular SLP schedule it will retain the duplication for 60 weeks. You have to create the SLP schedule like daily,weekly and monthly. If you have specific situation then create a different schedule but don't mix the retention otherwise you have unnecessary retention.


As I said, what I want to archieve is, that once a month a 6 weeks retention backups is duplicated to 60 weeks, and we can remove it from the library.

Just create a different schedule for this may be give it a name 60weeks.

[Mark_Solutions]

Incomplete SLP images usually have a retention of "forever" as that stops them expiring before their lifecycle is complete (i.e. have been duplicated)

My concern here is that you seem to be saying that you have a 41TB backlog of duplications - is that right? Do you know why?

Rather than duplicate and use SLPs (and if you only have tape and notmany drives that is a real issue for you) all you need are multiple schedules

Best way to do this is using calendar scheduling ...

A Monthly schedule that runs every every 1st Saturday of the month, has a retention of 60 weeks and uses the mothly volume pool - open window from 00:10 on Saturday to 23:50 on Saturday

A weekly schedule that runs every 2nd, 3rd, 4th, 5th Saturday of the month, has a retention of 6 weeks and uses the weekly volume pool - exactly the same open window as the Monthly one.

If you have a large backlog of duplications you will probably need to use:

nbstlutil cancel -lifecycle lifecyclename

to cancel thatback log after which you can look through all of what you currently have and just set a batch of them to be 60 weeks rather then 6 weeks to give a monthly set of tapes, even though they are on weekly tapes for that set.

Hope this makes sense and helps

[Volker_Spies1]

Hi, thanks, that was helpful, espacially "nbstlutil cancel -lifecycle lifecyclename" :) And yes I had a backuplog of 41TB, wich is weird because the server has approx. 6TB, why do I have such a large backlog? Only 1 backup was done wich the SLP in place. Currently, I have 3 schedules for that server, daily, weekly and month_week. Daily incremental 2 weeks retention. Weekly full 6 weeks retention month_week: The intention behind this backup is, that the I get a backup that has 6 weeks retention, to storage unit 1 and than do a copy with 60 weeks retention to storage unit 2. I thought SLP is the right solution for this. Is there a better way to do this. Volker

[Mark_Solutions]

If you do want two copies then an SLP is fine, as long as you have the resources to do the duplications efficiently.

I would make a new SLP with backup to STU1 and retention 6 weeks on weekly tapes then duplication to STU2 with retention 60 weeks on monthly tapes.

Next override the STU in the Monthly schedule with that SLP

[Volker_Spies1]

Hi Mark, that is exactly what I did and endet up with 41Tb of SLP backlog. ;) As I explained I have 3 schedules, day, week, month. In the month schedule the storage unit is overwritten with the SLP. In the SLP there is storage unit 1 with 6 weeks retention to the week pool and storage unit 2 with 6 week retention to the month pool. I have the resources to have two copies, the one with 6 weeks retention will stay in STU1 and the 60 week copy will be vaulted. But as I said, I've excatly the setup that you described, but ended up with that huge backlog. I have to say that the month schedule was set up with direct copy to two storage units 3-4 month ago, I edited the schedule to introduce SLP, can it be that the SLP tries to duplicate not just the backups taken with the SLP in the schedule, but also tries to duplicate older images taken weeks ago? Based on wich criteria are the images duplicated? I have a feeling that I get the 41TB (and growing) backlog because NBU tries to duplicate all backup images from this server with 6 weeks retention. Volker

[Mark_Solutions]

An SLP should not do that - it is image specific - even if you edit it old images retain their original SLP configurations.

So only backups that have ocurred since the SLP was introduced will have their images under the SLP control.

For a policy in question could you run the following and attach the output as text files just to that i can see what you have....(both in netbackup\bin\admincmd)

bppllist policyname -U

nbstl -L

Thanks

[Volker_Spies1]

Hi Mark, thanks for your help! Seems that NetBackup has a strong community! Here is the poilcy:Policy Name: P_WIN_FS_Sv42080OEL1183_FB Policy Type: FlashBackup-Windows Active: yes Effective date: 11/17/2011 13:51:52 File Restore Raw: yes Mult. Data Streams: yes Client Encrypt: no Checkpoint: no Policy Priority: 0 Max Jobs/Policy: Unlimited Disaster Recovery: 0 Collect BMR info: no Residence: sv42080oel1183_BU42080OEL0025 Volume Pool: OEL_P_DW_WIN_FS_ALL Server Group: *ANY* Keyword: (none specified) Data Classification: - Residence is Storage Lifecycle Policy: no Application Discovery: no Discovery Lifetime: 28800 seconds ASC Application and attributes: (none defined) Granular Restore Info: no Ignore Client Direct: no Enable Metadata Indexing: no Index server name: NULL Use Accelerator: no HW/OS/Client: Windows-x64 Windows2008 sv42080oel1183.emea.corp.geaag.com Include: \\.\E: \\.\G: \\.\H: \\.\I: \\.\J: \\.\K: Schedule: FULL_BACKUP_WEEK Type: Full Backup Maximum MPX: 16 Synthetic: 0 Checksum Change Detection: 0 PFI Recovery: 0 Retention Level: 10 (6 weeks) Number Copies: 1 Fail on Error: 0 Residence: (specific storage unit not required) Volume Pool: (same as policy volume pool) Server Group: (same as specified for policy) Calendar sched: Enabled Saturday, Week 2 Saturday, Week 3 Saturday, Week 4 Saturday, Week 5 Residence is Storage Lifecycle Policy: 0 Schedule indexing: 0 Daily Windows: Samstag 12:10:00 --> Samstag 16:10:00 Schedule: INCR_BACKUP Type: Cumulative Incremental Backup Maximum MPX: 16 Synthetic: 0 Checksum Change Detection: 0 PFI Recovery: 0 Retention Level: 1 (2 weeks) Number Copies: 1 Fail on Error: 0 Residence: (specific storage unit not required) Volume Pool: (same as policy volume pool) Server Group: (same as specified for policy) Calendar sched: Enabled Monday, Week 1 Tuesday, Week 1 Wednesday, Week 1 Thursday, Week 1 Friday, Week 1 Saturday, Week 1 Monday, Week 2 Tuesday, Week 2 Wednesday, Week 2 Thursday, Week 2 Friday, Week 2 Saturday, Week 2 Monday, Week 3 Tuesday, Week 3 Wednesday, Week 3 Thursday, Week 3 Friday, Week 3 Saturday, Week 3 Monday, Week 4 Tuesday, Week 4 Wednesday, Week 4 Thursday, Week 4 Friday, Week 4 Saturday, Week 4 Monday, Week 5 Tuesday, Week 5 Wednesday, Week 5 Thursday, Week 5 Friday, Week 5 Saturday, Week 5 EXCLUDE DATE 0 - 12/04/2012 EXCLUDE DATE 1 - 02/05/2013 Residence is Storage Lifecycle Policy: 0 Schedule indexing: 0 Daily Windows: Dienstag 01:00:00 --> Dienstag 06:00:00 Mittwoch 01:00:00 --> Mittwoch 06:00:00 Donnersta 01:00:00 --> Donnersta 06:00:00 Freitag 01:00:00 --> Freitag 06:00:00 Samstag 01:00:00 --> Samstag 06:00:00 Schedule: FULL_BACKUP_MONTH_WEEK Type: Full Backup Maximum MPX: 16 Synthetic: 0 Checksum Change Detection: 0 PFI Recovery: 0 Retention Level: 0 (1 week) Number Copies: 1 Fail on Error: 0 Residence: SV42080OEL1183_SLP_MONTH Volume Pool: (same as policy volume pool) Server Group: (same as specified for policy) Calendar sched: Enabled Saturday, Week 1 Residence is Storage Lifecycle Policy: 1 Schedule indexing: 0 Daily Windows: Samstag 12:10:00 --> Samstag 16:10:00 And the SLP: Name: SV42080OEL1183_SLP_MONTH Data Classification: (none specified) Duplication job priority: 0 State: active Version: 0 Operation 1 Use for: 0 (backup) Storage: sv42080oel1183_BU42080OEL0025 Volume Pool: OEL_P_DW_WIN_FS_ALL Server Group: Any Retention Type: 0 (Fixed) Retention Level: 10 (6 weeks) Alternate Read Server: (none specified) Preserve Multiplexing: false Enable Automatic Remote Import: false State: active Source: 0 (client) Operation ID: (none specified) Operation Index: 1 Operation 2 Use for: 1 (duplication) Storage: sv42080oel1183_BU42080OEL0026 Volume Pool: OEL_P_M_WIN_FS_ALL Server Group: Any Retention Type: 0 (Fixed) Retention Level: 11 (60 weeks) Alternate Read Server: (none specified) Preserve Multiplexing: true Enable Automatic Remote Import: false State: active Source: Operation 1 (backup:sv42080oel1183_BU42080OEL0025) Operation ID: (none specified) Operation Index: 2 There are some german words, but those are weekday names, if you need a translation, let me know. :) Volker

[Mark_Solutions]

Looks OK so cannot see why you should have an issue.

So if you run the following can you post the text file (as an attachment please)

nbstlutil list -lifecycle SV42080OEL1183_SLP_MONTH -image_incomplete

(unless you have already cancelled them all?)

[Volker_Spies1]

Unfortunately I canclled them all. :( But i can start a backup with the SLP in place and post you the results. Volker

[Mark_Solutions]

You could do to see if it gathers all the other images again - keep us posted

[Volker_Spies1]

HI Mark, Backup is running, I will keep you updated. I have 2 additional Questions regarding SLP I have a LIFECYCLE_PARAMETERS file in place, do I need to restart something on the master when I change parameters in that file? I have only big DB2 and Fileserver Backups (>600GB per Backup) to be duplicated so this is my Lifecycle file: MAX_MINUTES_TIL_FORCE_SMALL_DUPLICATION_JOB = 30 DUPLICATION_SESSION_INTERVAL_MINUTES = 5 MIN_GB_SIZE_PER_DUPLICATION_JOB = 1000 TAPE_RESOURCE_MULTIPLIER = 3 Do these Settings suit my needs? What does the TAPE RESOURCE_MULTIPLER do? I have 2 DSSUs in place, with staging schedules, there are lots of small files on these DSSUs. Are these duplication jobs affected by the lifecycle parameters file as well? Thanks Volker

[Mark_Solutions]

They are probably OK - the min gb size per duplic\ation is set to 1TB which is pretty high but this gets over rules by the 30 minutes force small duplication anyway

The tape multiplyer just allows more duplication jobs to go active - useful in a shared tape drive / multiple SLP environment

These settings only apply to SLPs and not to DSSU's

[Volker_Spies1]

Ok I tried the SLP with one Drive on the Fileserver again. That worked perfectly! The backup was ~376GB and took about 1:32 the duplication was exactly the same sze and took 1:06. These are values that I expected from the SLP I will wait until the next month backup kicks in to test if the SLP does work correctly then. But I have another question regarding SLP and multiplexing. I do copies of big DB2 systems with SLP. These backups have 8 streams. The backup took about 2:52 for 1.6TB. But the SLP seems not to keep the multiplexing at all, the 8 Streams result in 8 Duplication jobs and theses jobs run for about 3-4 hours wich is normal because the SLP has to read the whole tape for each job. But why does the SLP engage 8 Jobs and not only 1 Job that copies the whole 1.6TB and actually preserves the mutiplexing? When I understand the LIFECYCLE_PARAMETERS File correct, then I have to set the MIN_GB_SIZE_PER_DUPLICATION_JOB to a size bigger than the biggest backup I create to have the perserve multiplexing have its full effect? What I try to achieve is that I do a backup from our DB2 Database, wich are between 100Gb and 1.6TB in size and after that backup I need a copy with another retention. The copy shoeld not take longer than the backup itself. Any suggestion will be welcome. Volker

[Volker_Spies1]

Sorry Mark, didn't saw your reply, was typing my repley the same time.

Thanks for clarification. So when the tape multipler is 3 then I will have 3 active SLP jobs?

I chose to set the size to 1TB because the backups that should  be duplicated are from pretty big DB2 DBs.

But I have explained that in more detail in my post below.

Volker

[Mark_Solutions]

You cannot multiplex a duplication with NetBackup - it is by design

All duplications from disk to tape will be done one image at a time (whether it is basic, advanced or SLP duplications)

There is an advantahe in this in that your future restores from tape will be faster as they are not multiplexed

[Volker_Spies1]

Ah, ok!

hmm, this is a duplication from tape to tape, and why is there a preserve multiplexing checkbox in the SLP. I thought that NBU will be that smart to only read the tape once and copy it. So what is the purpose auf that setting?

I can see in the reports that the SLP is working image based. the images are >~200GB, so it would be smart to set my MIN_GB_SIZE to 200GB I guess.

Maybe SLP isn't the right way to do the duplications? I tried to write directly to two tapedrives, but I have a feeling that the actual data is read twice, because, when NBU does write to two tapes in parralell the backup time is nearly doubled.

Ok, thanks Mark! I think I have to ply a little but longer with the parameters and SLP to get the results I need.

Volker

[Mark_Solutions]

AH! forgot this was tape to tape ... that being the case is the multiplexing set the same for both Storage Units?

Open up the storage units and check that the Enable Mulitplexing is checked and the Maximum Streams per drive is the same for both

I am assuming that when the backups run they do all run at the same time so that the tapes are actaully multiplexed in the first place? Will only replicate the same as is on the original tape - not anything different.