Forum Discussion

terpigoryev's avatar
terpigoryev
Level 3
6 years ago

Useless incremental backups?

*** Moved to new post from https://vox.veritas.com/t5/NetBackup/backup-retention-with-incrementals/m-p/396768  *** Is it true that we always keep some useless differential incremental or cumulative...
  • Anshu_Pathak's avatar
    Anshu_Pathak
    6 years ago

    terpigoryev 

    #1 Incremental backups have relevant and important data which can be restored hence it is useful and required even without Full backups.

    #2 Only time when you can think of incremental as irrelevant without Full backup is when server (NetBackup client) has crashed and you want to perform a complete DR of that server to the point when last incremental was taken.

    Normally admins have retentions configured for full and incremental as per scenario #2. 

    Real issue is at the time of backup (it may seem strange :smileyhappy:) : During backup NetBackup checks for most recent successful Full backup of the client and time elapsed in seconds from that full backup while submitting an incremental backup. NetBackup client uses this elapsed time to figure out modified files since most recent Full backup. Now if there is no Full backup (recent or old) for this client, NetBackup will not be able to calculate elapsed time and you will start facing issues in incremental backups. It may also break accelerator or Synthetic backups.

    There are 2 types of NetBackup customers for these scenarios. First set (big/large numbers) of customer have implementation for scenario #2. In which Full backup is must and hence they have configured policy schedule retentions to meet that requirement. Second set of (small number) customers who see value in scenario #1.

    I completely understand your point to make incremental backup images dependent on Full and expire everthing (Full and corresponding incrementals) when full expires. This type of design will upset second set of customers and in some scenarios may cause data loss hence it is not implemented in NetBackup.

    So long story short,

    Is it true that we always keep some useless differential incremental or cumulative-incremental backups since full is expired? Yes, as NBU honors retentions configured in policy schedule for incremental backups rather than chains.

    And NBU doesn't track such chains anyhow? Yes it does NOT track such chains.

     

terpigoryev's avatar
terpigoryev
Level 3

sdo, thank you for your reply,

If I understand correctly, storing of incremental backups only depends on its retention period.

And if you want to expire all chain of incr backups when corresponding full backup is expired, you should manage it by retention period.

For example, 

Full schedule: weekly, at Sat, retention 14 days

Incr schedule 1: weekly, at Sun, retention 13 days

Incr schedule 2: weekly, at Mon, retention 12 days

Incr schedule 3: weekly, at Tue, retention 11 days

Incr schedule 4: weekly, at Wed, retention 10 days

Incr schedule 5: weekly, at Thu, retention 9 days

Incr schedule 6: weekly, at Fri, retention 8 days

So, often it is easier to have one incr schedule and to store several orphaned incremental backups.

sdo's avatar
sdo
Moderator
6 years ago

Many of us probably wouldn't try to do it that way.  Here's an example of what someone might do:

monthly schedule - full - retention one year or 13 months - calendar based, run on 1st Friday or Saturday - one long run window of e.g. Fri 18:00 to Mon 08:00

weekly schedule - full - retention one month or five weeks - calendar based, run on 2nd/3rd/4th/5th Friday or Saturday - one long run window of e.g. Fri 18:00 to Mon 08:00

daily schedule - cinc or diff - retention one month or five weeks - frequency based, frequency of one day (or I like to make the frequency = run window size + 1 hour... e.g. if the overnight window is 14 hours of 18:00 to 08:00 then I would make the frequency 15 hours) - five or six run windows each of e.g. 14 hours of 18:00 to next day 08:00

.

The trick with NetBackup is to not try to micro-manage it... yes, we need to understand how it works... but let NetBackup look after the expiry.  It's far less tedious admin work to have a few hundred policies of three schedules, than it is to have a few hundred (or more if you take micro-management to the nth degree) policies of eight schedules (monthly, weekly, six daily).

.

What happens then is that the trailing cinc/diff are still restorable from.  They are not completely orphaned unreachable / closed-off, i.e. one can still restore (changed files) from file-system based trailing incremental backups even if the preceding full has since expired.

The reason I prefer NetBackup's operating method of "expire the full by date" as opposed to BackupExec's operating method of "expire the full by chain" is that:

1) NetBackup then has highly consistent expiry, leading to highly consistent capacity management, leading to highly consistent storage and media management.

2) When backups are expected to expire and must expire they actually do expire - i.e. no problems with retaining backups that one is no longer allowed to retain.