Forum Discussion

tollboy's avatar
tollboy
Level 4
11 years ago

[Suggestion] Configuring Multiple Full schedules (Quaterly and monthly)

Hello,

I need a suggestion regarding schedules.  We have a NBU environment, where schedules are configured as follow.

Daily incremental, weekly cummulative incrementals (1 month), monthly full (3 monhts), Quaterly fulls (Infinity retentions). This is as per the business requirements.

Incrementals and cummulative incrementals are OK. But for the monthly and Quaterly fulls we have to maintain list of exclude and include dates.

On start of every quarter, quaterly fulls runs on those months no full backups should run. Vice versa is also true. So we have to keep populating exclude and include dates in our policies once they are exhausted.

This is how the policy schedule looks.

=================

 Schedule:          mon_mfull
    Type:            Full Backup
    Maximum MPX:     1
    Synthetic:       1
    PFI Recovery:    0
    Retention Level: 0 (1 week)
    Number Copies:   1
    Fail on Error:   0
    Residence:        (specific storage unit not required)
    Volume Pool:     (same as policy volume pool)
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Allowed to retry after run day
      SPECIFIC DATE 0 - 09/05/2011
      SPECIFIC DATE 1 - 11/07/2011
      SPECIFIC DATE 2 - 12/05/2011
      SPECIFIC DATE 3 - 02/06/2012
      SPECIFIC DATE 4 - 03/05/2012
      SPECIFIC DATE 5 - 05/07/2012
      SPECIFIC DATE 6 - 06/04/2012
      SPECIFIC DATE 7 - 08/06/2012
      SPECIFIC DATE 8 - 09/03/2012
      SPECIFIC DATE 9 - 11/05/2012
      SPECIFIC DATE 10 - 12/03/2012
      SPECIFIC DATE 11 - 02/04/2013
      SPECIFIC DATE 12 - 03/04/2013
      SPECIFIC DATE 13 - 05/06/2013
      SPECIFIC DATE 14 - 06/03/2013
      SPECIFIC DATE 15 - 08/05/2013
      SPECIFIC DATE 16 - 09/02/2013
      SPECIFIC DATE 17 - 11/04/2013
      SPECIFIC DATE 18 - 12/02/2013
      SPECIFIC DATE 19 - 02/04/2014
      SPECIFIC DATE 20 - 03/03/2014
      SPECIFIC DATE 21 - 05/05/2014
      SPECIFIC DATE 22 - 06/02/2014
      SPECIFIC DATE 23 - 08/04/2014
      SPECIFIC DATE 24 - 09/08/2014
      SPECIFIC DATE 25 - 11/03/2014
      SPECIFIC DATE 26 - 12/08/2014
      SPECIFIC DATE 27 - 02/02/2015
      EXCLUDE DATE 0 - 05/21/2012
    Residence is Storage Lifecycle Policy:     1
    Daily Windows:
          Tuesday    20:00:00  -->  Wednesday  07:00:00

  Schedule:          mon_qtr
    Type:            Full Backup
    Maximum MPX:     1
    Synthetic:       1
    PFI Recovery:    0
    Retention Level: 0 (1 week)
    Number Copies:   1
    Fail on Error:   0
    Residence:       (specific storage unit not required)
    Volume Pool:     (same as policy volume pool)
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      SPECIFIC DATE 0 - 05/07/2012
      SPECIFIC DATE 1 - 07/02/2012
      SPECIFIC DATE 2 - 10/08/2012
      SPECIFIC DATE 3 - 01/07/2013
      SPECIFIC DATE 4 - 04/08/2013
      SPECIFIC DATE 5 - 07/08/2013
      SPECIFIC DATE 6 - 10/07/2013
      SPECIFIC DATE 7 - 01/06/2014
      SPECIFIC DATE 8 - 04/07/2014
      SPECIFIC DATE 9 - 07/07/2014
      SPECIFIC DATE 10 - 10/06/2014
      SPECIFIC DATE 11 - 01/05/2015
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Monday     20:00:00  -->  Tuesday    07:00:00

  Schedule:          mon_wcumi
    Type:            Cumulative Incremental Backup
    Maximum MPX:     1
    Synthetic:       1
    PFI Recovery:    0
    Retention Level: 0 (1 week)
    Number Copies:   1
    Fail on Error:   0
    Residence:        (specific storage unit not required)
    Volume Pool:     (same as policy volume pool)
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Monday, Week 1
      Monday, Week 2
      Monday, Week 3
      Monday, Week 4
      Monday, Week 5
      EXCLUDE DATE 0 - 11/07/2011
      EXCLUDE DATE 1 - 12/05/2011
      EXCLUDE DATE 2 - 01/02/2012
      EXCLUDE DATE 3 - 02/06/2012
      EXCLUDE DATE 4 - 03/05/2012
      EXCLUDE DATE 5 - 04/02/2012
      EXCLUDE DATE 6 - 05/07/2012
      EXCLUDE DATE 7 - 05/21/2012
      EXCLUDE DATE 8 - 06/04/2012
      EXCLUDE DATE 9 - 07/02/2012
      EXCLUDE DATE 10 - 08/06/2012
      EXCLUDE DATE 11 - 09/03/2012
      EXCLUDE DATE 12 - 10/08/2012
      EXCLUDE DATE 13 - 11/05/2012
      EXCLUDE DATE 14 - 12/03/2012
      EXCLUDE DATE 15 - 01/07/2013
      EXCLUDE DATE 16 - 02/04/2013
      EXCLUDE DATE 17 - 03/04/2013
      EXCLUDE DATE 18 - 04/08/2013
      EXCLUDE DATE 19 - 05/06/2013
      EXCLUDE DATE 20 - 06/03/2013
      EXCLUDE DATE 21 - 07/08/2013
      EXCLUDE DATE 22 - 08/05/2013
      EXCLUDE DATE 23 - 09/02/2013
      EXCLUDE DATE 24 - 10/07/2013
      EXCLUDE DATE 25 - 11/04/2013
      EXCLUDE DATE 26 - 12/02/2013
      EXCLUDE DATE 27 - 01/06/2014
      EXCLUDE DATE 28 - 02/03/2014
      EXCLUDE DATE 29 - 03/03/2014
      EXCLUDE DATE 30 - 04/07/2014
      EXCLUDE DATE 31 - 05/05/2014
      EXCLUDE DATE 32 - 06/02/2014
      EXCLUDE DATE 33 - 07/07/2014
      EXCLUDE DATE 34 - 08/04/2014
      EXCLUDE DATE 35 - 09/08/2014
      EXCLUDE DATE 36 - 10/06/2014
      EXCLUDE DATE 37 - 11/03/2014
      EXCLUDE DATE 38 - 12/08/2014
      EXCLUDE DATE 39 - 01/05/2015
      EXCLUDE DATE 40 - 02/02/2015
    Residence is Storage Lifecycle Policy:     1
    Daily Windows:
          Monday     22:00:00  -->  Tuesday    07:00:00

  Schedule:          exmon_incr
    Type:            Differential Incremental Backup
    Frequency:       every 1 day
    Maximum MPX:     1
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 0 (1 week)
    Number Copies:   1
    Fail on Error:   0
    Residence:        (specific storage unit not required)
    Volume Pool:     (same as policy volume pool)
    Server Group:    (same as specified for policy)
      EXCLUDE DATE 0 - 05/21/2012
    Residence is Storage Lifecycle Policy:     1
    Daily Windows:
          Sunday     22:00:00  -->  Monday     07:00:00
          Tuesday    22:00:00  -->  Wednesday  07:00:00
          Wednesday  22:00:00  -->  Thursday   07:00:00
          Thursday   22:00:00  -->  Friday     07:00:00
          Friday     22:00:00  -->  Saturday   07:00:00
==================

 

We have 4 master servers and more then 800 policies,  Obviously its very tedious task. I want to get rid of this thing. I have the following idea in mind.

 

Create a seprate set of policy only for infinity quaterly backups and remove the quaterly schedules from the regular policy.

Now setup the monthly backups like "first monday of every month".  Same should be done with the quaterly infinity policy also.

During the regular times keep the infinity policy disabled and regular policy running. Durin the quaterly month in first week specifically enable the infinity policy and disable regular policy. (This can be done using simple script)

 

This is far beeter then what we are following  right now. If someone have other ideas please share them.

 

Thanks in advance!

 

(If I am not clear please let me know)

  • As far as the quarterly backups are concerned you'll probably have to go the Calendar schedule route & manually specify which dates the backups fall on.

    My experience has always been that if the start time on the backup windows are the same then NetBackup will only run the less frequent or higher retention backups if more than one schedule in the same policy is set to start at the same time.

    e.g. I would personally set up your policy with the following schedules:

    Daily Incremental - (exmon_incr) - as current but drop frequency to prevent 'schedule creep' and jobs not running if a manual backup is carried out (can drop 9 hours if required, 10 to be safe as window is 9 hours)

    Weekly Cumulative - (mon_wcumi) - change to frequency based with a 6 day frequency (or less - again can drop to 9 or 10 hours) ***change to Monday window

    Monthly Full - (mon_mfull) - calendar based. First Monday of month. Use same window (or at least start time) as mon_wcumi

    Quarterly Full - (mon_qtr) - as current. Use same window (or at least start time) as mon_wcumi

     

    If you schedule it correctly NetBackup *should* be clever enough to run exaactly what you want.

     

    *** Note: it seems as if your current set up for mon_mfull is incorrect - window opens Tuesday 2000hours but specific dates (except Feb 4th) are Mondays

    *** Edit: retention levels all 1 week?

7 Replies

  • You do know that you can create specific exclude/include lists down to the schedule level ?

    http://www.symantec.com/docs/HOWTO33491

    Format: exclude_list.POLICY_NAME.SCHEDULE_NAME.

    e.g.:

    exlude_list.UNIX_FS.daily

    exlude_list.UNIX_FS.weekly

    If you use frequency based backup you can define both daily,weekly, quaterly backup in the same policy and Netbackup will not run both a daily and weekly/monthly/quaterly backup at the same time for a given client. 

     

  • Hello Nicolai,

     

    Thanks for the reply, but  the link you given, I gues it is for excluding backup selectiong. I am looking for excluding dates.

    Also frequency based schedules have hours, days and weeks. For quatery if I give 12 weeks, then it will be mess, as number of week changes in some months.

     

     

     

  • As far as the quarterly backups are concerned you'll probably have to go the Calendar schedule route & manually specify which dates the backups fall on.

    My experience has always been that if the start time on the backup windows are the same then NetBackup will only run the less frequent or higher retention backups if more than one schedule in the same policy is set to start at the same time.

    e.g. I would personally set up your policy with the following schedules:

    Daily Incremental - (exmon_incr) - as current but drop frequency to prevent 'schedule creep' and jobs not running if a manual backup is carried out (can drop 9 hours if required, 10 to be safe as window is 9 hours)

    Weekly Cumulative - (mon_wcumi) - change to frequency based with a 6 day frequency (or less - again can drop to 9 or 10 hours) ***change to Monday window

    Monthly Full - (mon_mfull) - calendar based. First Monday of month. Use same window (or at least start time) as mon_wcumi

    Quarterly Full - (mon_qtr) - as current. Use same window (or at least start time) as mon_wcumi

     

    If you schedule it correctly NetBackup *should* be clever enough to run exaactly what you want.

     

    *** Note: it seems as if your current set up for mon_mfull is incorrect - window opens Tuesday 2000hours but specific dates (except Feb 4th) are Mondays

    *** Edit: retention levels all 1 week?

  • Sorry - I mis-read the post. I red it as include and excludes lists.

  • Andy,

    Thanks for pointing out that mistake, I will rectify it.

    Retention: Every schedule goes to different different SLPs, I had changed the residence of the schedules before posting here. wink SLP manages the retention.

    Schedule: As per your suggested configuration, We will get rid from maintaining the date list for Weeklly Cumulatives (mon_wcumi) but we still have to maintain the exclude date and specific date for Monthly Full (mon_mfull) and Quaterly Full (mon_qtr)? Or  I am getting it wrong?

     

  • You *should* only need to specify the dates that you want the quarterly backups to run.

    There *shouldn't* be any requirement for exclude dates.

    We used to do the same thing - but not quarterlies - with include and exclude dates where monthly calendar-based and weekly frequency-based backups clashed but found that having the same start time made this redundant.

    You *may* want to monitor very closely to start with as there *may* be some tweaking required (e.g. advance the start time of the monthly/quarterly backups.

    You may notice a lot of "*...*" type emphasis above for should/shouldn't/may - it's not a definite science but it can (and has been proved to) work.

    As mentioned earlier, we didn't have 2 lots of calendar-based schedules contending start times in our policies - maybe this will throw a spanner in the works, I don't know.

     

    For quatery if I give 12 weeks, then it will be mess, as number of week changes in some months.

    I presume, therefore, that you are tied to running your monthly/quarterly backups on a strict calendar month basis.

    An article we used to quote that Marianne got David Chapa to 're-publish' for us:

    https://www-secure.symantec.com/connect/articles/netbackup-frequency-based-scheduling-1

  • Thanks for your valuable suggestion Andy,

    I will test this on few policies in our environment and see how it goes.

    Yes, the environment is configured to run full backups on first full week of the month (if 1st is anything other then monday then its not a full week). Then different policies runs on different day of the week. Like the one we are taking as example, for this one full/cummulative runs on Monday, same way policies are spread across whole week.

    Thanks again for the article as well, I will go thorugh them and see if we can make any use of it.