Highlighted

Useless incremental backups?

*** Moved to new post from https://vox.veritas.com/t5/NetBackup/backup-retention-with-incrementals/m-p/396768  ***

Is it true that we always keep some useless differential incremental or cumulative-incremental backups since full is expired?

And NBU doesn't track such chains anyhow?

1 Solution

Accepted Solutions
Highlighted
Accepted Solution!

Re: Useless incremental backups?

@terpigoryev 

#1 Incremental backups have relevant and important data which can be restored hence it is useful and required even without Full backups.

#2 Only time when you can think of incremental as irrelevant without Full backup is when server (NetBackup client) has crashed and you want to perform a complete DR of that server to the point when last incremental was taken.

Normally admins have retentions configured for full and incremental as per scenario #2. 

Real issue is at the time of backup (it may seem strange Smiley Happy) : During backup NetBackup checks for most recent successful Full backup of the client and time elapsed in seconds from that full backup while submitting an incremental backup. NetBackup client uses this elapsed time to figure out modified files since most recent Full backup. Now if there is no Full backup (recent or old) for this client, NetBackup will not be able to calculate elapsed time and you will start facing issues in incremental backups. It may also break accelerator or Synthetic backups.

There are 2 types of NetBackup customers for these scenarios. First set (big/large numbers) of customer have implementation for scenario #2. In which Full backup is must and hence they have configured policy schedule retentions to meet that requirement. Second set of (small number) customers who see value in scenario #1.

I completely understand your point to make incremental backup images dependent on Full and expire everthing (Full and corresponding incrementals) when full expires. This type of design will upset second set of customers and in some scenarios may cause data loss hence it is not implemented in NetBackup.

So long story short,

Is it true that we always keep some useless differential incremental or cumulative-incremental backups since full is expired? Yes, as NBU honors retentions configured in policy schedule for incremental backups rather than chains.

And NBU doesn't track such chains anyhow? Yes it does NOT track such chains.

 

View solution in original post

8 Replies
Highlighted

Re: Useless incremental backups?

The word "useless" is a bit harsh.  And incrementals are tracked, so to speak, as they are saved versus previous full.  This point about trailing incrementals all depends upon your perspective.  What I can say is that it certainly pays to know, to understand, how NetBackup works.  IMO, the trailing incrementals by retention period are nothing more than a product feature.  It has proved useful before, and it will prove useful again.  It's never bothered me, or my management, as long as one understands the benefits versus implications, and I've been working with NetBackup for over 15 years.  It's just a case of that's the way it works, to have trailing incrementals. And it is certainly an aspect of application behaviour that one really should take in to consideration for one's retention scheduling and capacity management.  The simple truth of the matter is that not all backup applications behave the same.  They never will.  They are all different to each other.

Highlighted

Re: Useless incremental backups?

sdo, thank you for your reply,

If I understand correctly, storing of incremental backups only depends on its retention period.

And if you want to expire all chain of incr backups when corresponding full backup is expired, you should manage it by retention period.

For example, 

Full schedule: weekly, at Sat, retention 14 days

Incr schedule 1: weekly, at Sun, retention 13 days

Incr schedule 2: weekly, at Mon, retention 12 days

Incr schedule 3: weekly, at Tue, retention 11 days

Incr schedule 4: weekly, at Wed, retention 10 days

Incr schedule 5: weekly, at Thu, retention 9 days

Incr schedule 6: weekly, at Fri, retention 8 days

So, often it is easier to have one incr schedule and to store several orphaned incremental backups.

Highlighted

Re: Useless incremental backups?

Many of us probably wouldn't try to do it that way.  Here's an example of what someone might do:

monthly schedule - full - retention one year or 13 months - calendar based, run on 1st Friday or Saturday - one long run window of e.g. Fri 18:00 to Mon 08:00

weekly schedule - full - retention one month or five weeks - calendar based, run on 2nd/3rd/4th/5th Friday or Saturday - one long run window of e.g. Fri 18:00 to Mon 08:00

daily schedule - cinc or diff - retention one month or five weeks - frequency based, frequency of one day (or I like to make the frequency = run window size + 1 hour... e.g. if the overnight window is 14 hours of 18:00 to 08:00 then I would make the frequency 15 hours) - five or six run windows each of e.g. 14 hours of 18:00 to next day 08:00

.

The trick with NetBackup is to not try to micro-manage it... yes, we need to understand how it works... but let NetBackup look after the expiry.  It's far less tedious admin work to have a few hundred policies of three schedules, than it is to have a few hundred (or more if you take micro-management to the nth degree) policies of eight schedules (monthly, weekly, six daily).

.

What happens then is that the trailing cinc/diff are still restorable from.  They are not completely orphaned unreachable / closed-off, i.e. one can still restore (changed files) from file-system based trailing incremental backups even if the preceding full has since expired.

The reason I prefer NetBackup's operating method of "expire the full by date" as opposed to BackupExec's operating method of "expire the full by chain" is that:

1) NetBackup then has highly consistent expiry, leading to highly consistent capacity management, leading to highly consistent storage and media management.

2) When backups are expected to expire and must expire they actually do expire - i.e. no problems with retaining backups that one is no longer allowed to retain.

Highlighted

Re: Useless incremental backups?

Full backup and incremental should have different life time - retain full backup at least same retention as incremental + "some air" in case backup is failed over a longer period.

E.g:
Incremental = 14 days
Fulls = 3 weeks.

Then incremental never miss their full backup reference.

Highlighted

Re: Useless incremental backups?


@Nicolai wrote:

Full backup and incremental should have different life time - retain full backup at least same retention as incremental + "some air" in case backup is failed over a longer period.

E.g:
Incremental = 14 days
Fulls = 3 weeks.

Then incremental never miss their full backup reference.


This is for Cumulative only? As I understand differential incrementals should have longer retention period than full backups. That cause trailing backups.


Highlighted
Accepted Solution!

Re: Useless incremental backups?

@terpigoryev 

#1 Incremental backups have relevant and important data which can be restored hence it is useful and required even without Full backups.

#2 Only time when you can think of incremental as irrelevant without Full backup is when server (NetBackup client) has crashed and you want to perform a complete DR of that server to the point when last incremental was taken.

Normally admins have retentions configured for full and incremental as per scenario #2. 

Real issue is at the time of backup (it may seem strange Smiley Happy) : During backup NetBackup checks for most recent successful Full backup of the client and time elapsed in seconds from that full backup while submitting an incremental backup. NetBackup client uses this elapsed time to figure out modified files since most recent Full backup. Now if there is no Full backup (recent or old) for this client, NetBackup will not be able to calculate elapsed time and you will start facing issues in incremental backups. It may also break accelerator or Synthetic backups.

There are 2 types of NetBackup customers for these scenarios. First set (big/large numbers) of customer have implementation for scenario #2. In which Full backup is must and hence they have configured policy schedule retentions to meet that requirement. Second set of (small number) customers who see value in scenario #1.

I completely understand your point to make incremental backup images dependent on Full and expire everthing (Full and corresponding incrementals) when full expires. This type of design will upset second set of customers and in some scenarios may cause data loss hence it is not implemented in NetBackup.

So long story short,

Is it true that we always keep some useless differential incremental or cumulative-incremental backups since full is expired? Yes, as NBU honors retentions configured in policy schedule for incremental backups rather than chains.

And NBU doesn't track such chains anyhow? Yes it does NOT track such chains.

 

View solution in original post

Highlighted

Re: Useless incremental backups?

Hello,

one note:

- in a case of file/NAS/VM backups, an incremental backup without initial full can be still used for file-level recoveries - for files which were changed at the time between these incremental backups

- for compelete file/NAS/VM Disaster Recovery, yes these backups are useless

- for database recoveries (Oracle, MSSQL, etc.) these backups are useless, too

Regards

Michal

 

Highlighted

Re: Useless incremental backups?

Hi people

Thank you very much for replies! I suppose the topic is fully clarified.

I'm at a loss as to what answer to mark as a solution, really)