cancel
Showing results for 
Search instead for 
Did you mean: 

Run backup Full in the incremental backup schedule. Policies with Deduplication

Heron_Batista
Level 4

 

Run full backup on the weekend and incremental from Monday to Friday.
Use deduplication in policies.

The schedule "Full_Mensal" run in MSDP disk and copy to tape (duplication) SLP.

 

--> Why Monday's full backup runs? It is set "Incremental"!!!

 

Thanks.

 

1 ACCEPTED SOLUTION

Accepted Solutions

Mark_Solutions
Level 6
Partner Accredited Certified

The best way to get good dedupe is to have a good set of data for new backups to dedupe against.

If you expire after copy then other backups will have a poor start and your dedupe figures could be quite low

You will also put your system under heavy load as it has to clear down all that data every day

If you want it to work to its best give it a decentretention period .. you will get better dedupe, faster accelerated backups and a less overworked system

I would suggest looking at your systems setup and modifying it .. expire after copy makes no sense for a dedupe system

View solution in original post

28 REPLIES 28

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Are schedules in same or different policies? Full and Incremental schedules must be in the same policy. Was the weekend Full in same policy successful?

Heron_Batista
Level 4

 

Schedules are the same policies.

Yes, back up the weekend was successful. see: (SLP, disk and duplication to tape).

it happens to all policies that use deduplication.

 

bppllist:

 

==================================
Policy Name:       P_WIN_FS_P_DEDUP04
Options:           0x0
template:          FALSE
audit_reason:         ?
Names:             (none)
Policy Type:       MS-Windows (13)
Active:            yes
Effective date:    02/03/1991 17:25:54
Client Compress:   no
Follow NFS Mnts:   no
Backup netwrk drvs:no
Collect TIR info:      no
Mult. Data Stream: no
Perform Snapshot Backup:   no
Snapshot Method:           (none)
Snapshot Method Arguments: (none)
Perform Offhost Backup:    no
Backup Copy:               0
Use Data Mover:            no
Data Mover Type:           -1
Use Alternate Client:      no
Alternate Client Name:     (none)
Use Virtual Machine:      0
Hyper-V Server Name:     (none)
Enable Instant Recovery:   no
Policy Priority:   0
Max Jobs/Policy:   Unlimited
Disaster Recovery: 0
Collect BMR Info:  no
Keyword:           (none specified)
Data Classification:       -
Residence is Storage Lifecycle Policy:    no
Client Encrypt:    no
Checkpoint:        yes
      Interval:    60
Residence:         nbumedia04_disk_dedup
Volume Pool:       Pool_Win
Server Group:      PRODEMGE
Granular Restore Info:  no
Exchange Source attributes:              no
Exchange DAG Preferred Server: (none defined)
Application Discovery:      no
Discovery Lifetime:      0 seconds
ASC Application and attributes: (none defined)
Generation:      53
Ignore Client Direct:  yes
Use Accelerator:  yes
Optimized Backup:  no
Client/HW/OS/Pri/CIT:  cainffss1b Windows-x64 Windows2008 0 0 0 ?
Include:           ALL_LOCAL_DRIVES
Schedule:              Full_Mensal
  Type:                FULL (0)
  Calendar sched: Enabled
   Included Dates-----------
      Saturday, Week 5
   Excluded Dates----------
      No specific exclude dates entered
      No exclude days of week entered
  Retention Level:     0 (1 week)
  u-wind/o/d:          0 0
  Incr Type:           DELTA (0)
  Alt Read Host:       (none defined)
  Max Frag Size:       0 MB
  Synthetic:           0
  Checksum Change Detection: 0
  PFI Recovery:        0
  Maximum MPX:         1
  Number Copies:       1
  Fail on Error:       0
  Residence:           SLP_WIN_DUPLICA04_MENSAL
  Volume Pool:         (same as policy volume pool)
  Server Group:        (same as specified for policy)
  Residence is Storage Lifecycle Policy:         1
  Daily Windows:
   Day         Open       Close       W-Open     W-Close
   Sunday      000:00:00  000:00:00
   Monday      016:00:00  016:00:00
   Tuesday     000:00:00  000:00:00
   Wednesday   000:00:00  000:00:00
   Thursday    000:00:00  000:00:00
   Friday      000:00:00  000:00:00
   Saturday    008:30:00  064:00:00   152:30:00  208:00:00 040:00:00
Schedule:              Full_Semanal
  Type:                FULL (0)
  Calendar sched: Enabled
   Included Dates-----------
    SPECIFIC DATE 0 - 01/23/2016
    SPECIFIC DATE 1 - 04/23/2016
    SPECIFIC DATE 2 - 07/23/2016
    SPECIFIC DATE 3 - 10/22/2016
    SPECIFIC DATE 4 - 12/24/2016
      Saturday, Week 1
      Saturday, Week 2
      Saturday, Week 3
   Excluded Dates----------
      No specific exclude dates entered
      No exclude days of week entered
  Retention Level:     0 (1 week)
  u-wind/o/d:          0 0
  Incr Type:           DELTA (0)
  Alt Read Host:       (none defined)
  Max Frag Size:       0 MB
  Synthetic:           0
  Checksum Change Detection: 0
  PFI Recovery:        0
  Maximum MPX:         1
  Number Copies:       1
  Fail on Error:       0
  Residence:           SLP_DUPLICA04_SEMANAL
  Volume Pool:         (same as policy volume pool)
  Server Group:        (same as specified for policy)
  Residence is Storage Lifecycle Policy:         1
  Daily Windows:
   Day         Open       Close       W-Open     W-Close
   Sunday      000:00:00  000:00:00
   Monday      000:00:00  000:00:00
   Tuesday     000:00:00  000:00:00
   Wednesday   000:00:00  000:00:00
   Thursday    000:00:00  000:00:00
   Friday      000:00:00  000:00:00
   Saturday    008:00:00  064:00:00   152:00:00  208:00:00 040:00:00
Schedule:              Incremental
  Type:                INCR (1)
  Calendar sched: Enabled
   Included Dates-----------
      Monday, Week 1
      Tuesday, Week 1
      Wednesday, Week 1
      Thursday, Week 1
      Friday, Week 1
      Monday, Week 2
      Tuesday, Week 2
      Wednesday, Week 2
      Thursday, Week 2
      Friday, Week 2
      Monday, Week 3
      Tuesday, Week 3
      Wednesday, Week 3
      Thursday, Week 3
      Friday, Week 3
      Monday, Week 4
      Tuesday, Week 4
      Wednesday, Week 4
      Thursday, Week 4
      Friday, Week 4
      Monday, Week 5
      Tuesday, Week 5
      Wednesday, Week 5
      Thursday, Week 5
      Friday, Week 5
   Excluded Dates----------
      No specific exclude dates entered
      No exclude days of week entered
  Retention Level:     0 (1 week)
  u-wind/o/d:          0 0
  Incr Type:           DELTA (0)
  Alt Read Host:       (none defined)
  Max Frag Size:       0 MB
  PFI Recovery:        0
  Maximum MPX:         1
  Number Copies:       1
  Fail on Error:       0
  Residence:           SLP_DUPLICA04_INCREMENTAL
  Volume Pool:         (same as policy volume pool)
  Server Group:        (same as specified for policy)
  Residence is Storage Lifecycle Policy:         1
  Daily Windows:
   Day         Open       Close       W-Open     W-Close
   Sunday      000:00:00  000:00:00
   Monday      020:00:00  032:00:00   044:00:00  056:00:00
   Tuesday     020:00:00  032:00:00   068:00:00  080:00:00
   Wednesday   020:00:00  032:00:00   092:00:00  104:00:00
   Thursday    020:00:00  032:00:00   116:00:00  128:00:00
   Friday      020:00:00  032:00:00   140:00:00  152:00:00
   Saturday    000:00:00  000:00:00
 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Please show us output of backups since the start of Specific date in schedule:

bpimagelist -client cainffss1b -d 01/23/2016 -U

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Look at the size of the full and differentials - why do you think fulls are done during the week?

It looks a LOT smaller than the fulls, right?

Heron_Batista
Level 4

ops!!! sorry.
see the above post (edited)

 

Heron_Batista
Level 4

see:

Capturar.JPG

 

It can not see here, because the only Differential Incremental backup run full is what runs after the schedule "Full_Mensal"
and is running now.

see, active monitor:

Capturar_monitor.JPG

 

 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
I still don't understand what the problem is. The Diff is running as per the schedule. Yes, the current backup size seems bigger than previous Diff backups, but this is hardly ever a NBU issue. Windows backups are by default running Incrementals based on Archive bit. If you do a Google search, you will see that there are many OS and application-related events that could reset archive bit. Not only backups.

sdo
Moderator
Moderator
Partner    VIP    Certified
If someone renamed the top of a big tree, or moved a big folder tree, or changed perms on a whole tree, or you removed client and re-added, or drive removed from policy and re-added, all result in what can appear to be, or in fact are, effective fulls.

Heron_Batista
Level 4

Hi Marianne e sdo,


I understand there is a high modification of files, effective full, but is not it. The case is curious.
I did see a report of the last months

all backup "Incremental" after a "Full_Mensal" Protected Data Size is full backup.

 

relat.JPG

Mark_Solutions
Level 6
Partner Accredited Certified

What does the Detail Status of teh currently running backup job say?

Any clues in there perhaps saying that it has issues and is running a full backup of a drive?

It is a very large backup which uses accelerator so maybe the track log is having problems .. or even being disrupted by Anti Virus / lack of disk space

sdo
Moderator
Moderator
Partner    VIP    Certified

Perhaps, if the client is also using "NTFS Change Journal" tracking, and then if the USN loops, then maybe this is causing an effective full.

Track your journal size using:

https://technet.microsoft.com/en-gb/library/cc788042.aspx

.

Or, maybe a combination of Accelerator issues and a looping USN, or corrupt CJ.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
So, on certain days, the Diff backup is running as full. The next day we see a small Diff again. Are you keeping track of policy changes? Certain policy changes e.g. changes to Backup Selection or selecting/deselecting Accelerator can trigger a full backup.

sdo
Moderator
Moderator
Partner    VIP    Certified

The INCR schedule uses an SLP named SLP_DUPLICA04_INCREMENTAL.  In this SLP, is the storage unit of the backup stage a Storage Unit Group containing two or more de-dupe pool storage units?

If "yes", then read on...

Accelerator may have no previous full on the storage unit (which got selected at backup job run time), and so the incremental is an effective full.

When one has multiple MSDP media servers, then there is no way of establishing "affinity" of a backup client to an MSDP storage unit.

Remember, de-dupe is not global to a NetBackup domain, instead de-dupe is local to an MSDP.

The way around this "artifact" of increased incremental size... is to use multipe storage unit groups, but each storage unit group has only one MSDP storage unit in it, so that backups always go to the same storage unit - most of the time.  And so, when an MSDP is down for maintenance, then all one has to do... is temporarily change the storage unit in the storage unit group, and then all backup schedules and SLP backup stages will use the new temporary "single" MSDP pool.

i.e. balance your backup "workload" by backup jobs size and run day, across your multiple MSDP storage units, by working out which Storage Unit Unit they should use.  The trick here is to have quite similar names for your actual storage units and the name of your notional storage unit group linkage - so that they are easy to manage and easier to understand the configuration.  Do you need this explained in more detail?

HTH.

sdo
Moderator
Moderator
Partner    VIP    Certified

I once had to balance 6000 backup clients from about 700 backup policies across 14 MSDP media servers... to avoid exactly your problem.  Nasty job.  And a similar excercise at several other NetBackup domains.  Necessity being the mother of invention, so I wrote a script to do it.   i.e. inspect the backup policies, match up the schedules and SLPs to the actual "workload of the backup day" and then it would produce a script containing the necessary policy modification commands so that, in future, all backup clients were evenly spread across media servers MSDP pools - i.e. establish "affinity" between a client and an MSDP pool.

Heron_Batista
Level 4

Hi,

* There were no changes in policies/clients.
* SLP has only one de-dupe units storage pool.
* Have two polls MSDP, and are balanced.


I'm thinking, I think I know the problem;
The policies are configured to backup to a MSDP pool. But the schedule "Full_Mensal" has SLP duplication to tape, and the backup "expires after copy". see the image.

slp.JPG

 

The Netbackup is dumb.  The Schedule "Incremental" is run full backup because it does not have the full last on disk.
I will test it !!!

 

sdo
Moderator
Moderator
Partner    VIP    Certified

Is it dumb?  Accelerator won't have a position on which to base the incremental, so it takes a full.   NetBackup is trying its best to save your data the safest way it can.  Then again, you should be able to avoid the circumstances (and I say circumstance, because it's not a real problem as such) by turning off accelerator.

Mark_Solutions
Level 6
Partner Accredited Certified

The best way to get good dedupe is to have a good set of data for new backups to dedupe against.

If you expire after copy then other backups will have a poor start and your dedupe figures could be quite low

You will also put your system under heavy load as it has to clear down all that data every day

If you want it to work to its best give it a decentretention period .. you will get better dedupe, faster accelerated backups and a less overworked system

I would suggest looking at your systems setup and modifying it .. expire after copy makes no sense for a dedupe system

sdo
Moderator
Moderator
Partner    VIP    Certified

If you were to do as Mark suggests, AND enable client side dedupe on the 5TB client, then much much less data would be transfered via LAN and far less work for the media server to do in terms of hashing the blocks to make fingerprints, i.e. the media server can simply concentrate on saying back to the client... "Yes client, I have that already, and yes I'll make a note that I now have more than one use for that finger-printed block.".

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

What is the point of dedupe when you expire after duplication?

You will save yourself a lot of time, money and hassle if you backup directly to tape.