Netbackup 7.5, Client Dedupliction and Accelerator

[cds171612]

I have been running Netbackup 7.1 for about 80 remote offices with Client Dedupliction.

Initially a manual full and then once seeded:

Mon-fri = Differential Incremental

Saturday= Synthetic Full

Now with 7.5 I see the new accelerator option and just looking for advice on what changes to make? Enable accelerator on all my policies and then instead of doing a synthetic full on Saturdays should I do an accelerator full on Friday nights? Or do I just keep everything the way it is but enable the accelerator option to further speed up my differential incrementals?

 Thanks for any input/advice in advance.

[Yasuhisa_Ishika]

Sythetic backup does not accutually take backup at a tome, whereas accelerator backups read backup target at a time. So, if you would like to keep current RPO, and would like to avoid backup load at Saturday, you should change Differential backup at Frighday night to accelerator full backup. Otherwise, if you don't mind load at Saturday, you may configure accelerator full at Saturday. Note that Change Journal does not support multiple policy on single client. I would configure backups with weekly force rescan, and would not enable change journal on Windows clients.

[cds171612]

Thanks for the comments, once I configured a test policy I realized you can't use accelerator with a policy that has Synthetic backups so there goes that.

I ended up doing Accelerator diffferential Incrementals Mon-Thurs and then a Full Accelerator on Friday nights.

Now your comment about using "Force rescan" on a weekly basis I am curious about. One of the reasons we use a Full Sythentic is that some of these remote backups are 2-3 TB in size and even to rerun a fully seeded full can potentially take a week or 2.

I will test this out, but is an Accelerator Force Rescan still faster than a normal full backup? (keep in mind Client deduplication is enabled)

[Stanleyj]

Does accelerator actually work with incrementals?  I have been curious about that because a full accelerator backup is "suppose" to be similar to an incremtal because it is only sending the changes across the wire and is then repackaged (like a synthetic) to be a full.  They claim this full accelerator backup should only take the time of an incremental.

Im no where near the scale of TBs accross the WAN but our remote locations are pushing around 20gb for a full and running on cable and dsl lines.

In my testing a full regular backup took 11 hours for 12gbs.  The next day i changed 5gb worth of data and then ran an accelerator full and it took 45mins. I was pretty impressed considering it was a full. 

I have not tested an incremental but i was thinking that the accelerator sort of replaced those. 

The rescan function I have not tested either but according to the admin guide it will basically do a regular full backup to set a new base.

Let me know how the incrementals in combination with accelerator are working for you.  Thanks.

[cds171612]

Yes an accelerator does indeed work with incrementals...and I would recommend it too.  Otherwise your catalogue size will increase substantially over time.

Right now I have my remote clients configured using Client Deduplication, TIR, Accelerator and the following schedule:

Mon-Thurs = differential incremental

Fri = Full

First Friday of month = Force Rescan Full (*assuming this still runs faster than a normal full which I think it does)

So far it has been working for about 2 weeks quite well on a wide range of clients from 8gb to 2TB.

Will report back once I have a full month or so of this running and if I made any tweaks.

[Stanleyj]

I appreiciate the info.  The schedules for my normal backups are setup similar to that today so i think i wil setup a few of these remote locations and see what happens.

TIR is a little confusing to understand so i have never actually turned it on.   Do you see any benifits from it?

[cds171612]

The main reason I used it was with Incrementals and Full Synthetic backups. Basically if someone purposely deletes or moves a folder during the week and you do a restore from a backup without TIR enabled, your restore would bring back that file or folder in both locations...with TIR it tracks where moved/deleted files/folders should be restored to(if at all).

Now with Accelerator, you are actually running Full backups so its not a requirement..however if a full restore was needed on a folder from say Wednesday, you would need to restore from the last full + the incrementals so I think you would want TIR enabled so that the exact structure is restored.

[Mark_Solutions]

If you are using client side de-dupe then even a Forced Full should only send minimal data across the link and shouldn't take that long?

Dont forget to turn on compression (in the pd.conf) on the client to help further.

[Stanleyj]

Does anyone know if the tracking folder and the change journal data are purged after successful backups?

Just wondering if these are directories that will continue to grow or if they stayed "cleaned up".

[Stanleyj]

I just created a policy to use Accelerator, TIR and change journal.  In the details of the job it says "NOT using change journal data for <>: Not supported with true image backups"

I found an article saying that the change journal cannot be used with TIR. 

For accelerator to work at top speed i was under the impression that the change journal option was needed.

Above you said it would be benifical to turn on TIR along with accelerator but is the impacting performance now?

[cds171612]

So far having TIR enabled (and change journals disabled) I have been getting pretty awesome backup speeds, basically like it used to be in the Puredisk 6.,6 days.

So I don't think it has a major impact on performance, so far I am very pleased. Still waiting to see how well it runs on the WAN for a 2TB + office, with accelerator forced rescan enabled as well... . I have that set to run once a month, will know better how it does in just over a week when it's scheduled to run.

[rbkguy]

I run a very simular setup here and it works great. One of the biggest problems I had was transfering the data from our eight remote sites to our head office over 10MB pipes into a 25MB pipe in head office without effecting network activity during business hours.

For anyone that has over 500GB of data to transfer to dedup pool over slow LAN\WAN links is to backup the data to tape or disk and send a phyiscal copy of the data to your office and duplicate the data into the dedup pool before your first run.It will save you days to weeks if you have slow LAN\WAN links!!!

[AbdulRasheed · Accepted solution]

Hi guys,

  There is really no need for old school Optimized Synthetic schedules if you are sending backups to a storage unit that supports Accelerator. As I explained in my blog...

Can you compare and contrast NetBackup Accelerator and NetBackup Optimized Synthetic backup?

NetBackup Accelerator provides all the values you get from NetBackup Optimized Synthetic backup and a lot more. In fact, if you have a supported storage pool for NetBackup Accelerator, there is really no need for you to use NetBackup Optimized Synthetic backup.

NetBackup Optimized Synthetic backup is a post-backup synthesis. You need a separate schedule that generates synthetic backup after incremental backups are done.  NetBackup Accelerator generates the full backup inline when data is being sent from client.

NetBackup Optimized Synthetic backup requires you to run traditional incremental backups. Hence all the limitations of a traditional incremental backup are applicable. For example, the incremental backups do require NetBackup client to enumerate the entire file system to identify the changes. NetBackup Accelerator makes it possible to intelligently identify changes and read just the changed files.

I can list a lot more, but you get the point by now. Bottom line is… if your storage pool supports NetBackup Accelerator, there is no need to use the older NetBackup Optimized Synthetic backup schedules.

[AbdulRasheed]

Accelerator works with both full and incremental schedules. Please see this FAQ

NetBackup Accelerator provides full backups for the cost of an incremental backup.

cost reduction in full backups = reduction in backup window, backup storage, client CPU, client memory, Client disk I/O, network bandwidth etc.

The primary benefit from NetBackup Accelerator is for full backups. However, NetBackup Accelerator also reduces a subset of costs in running incremental backups. 

cost reduction in incremental backups = reduction in client CPU, client memory, Client disk I/O 

[AbdulRasheed]

Q) Since I can get full backups for the cost of doing an incremental backup, should I simply delete my incremental schedules and increase the frequency of full backups? 

A) Not recommended unless you are not concerned about the catalog growth. Note that full backups will have catalog files (the "*.f" files in NetBackup image database) larger than those of incremental backups. Running full backups instead of current incremental backups would mean that your image catalog size would increase. Larger catalog requires more space on your master server and it takes longer to run catalog backups. 

As I mentioned in the answer to the previous question, NetBackup Accelerator does help with incremental backups as well by significantly reducing the impact of backups on client's resources. Stick with your current schedules and take advantage of NetBackup Accelerator. 

[AbdulRasheed]

The size of the track log is a function of number of files and size of the files in the file system. The size of the track log does not increase with increase in data change rate in the file system. In other words, if the same file is getting modified, it does not increase the size of track log. In fact, this is the beauty of track log over traditional kernel level file system change journals. 

The track log is persisted on client. This track log is a self managed and resizes itself as the parameters discussed above changes. There is no clean up required as it is not an ever growing 'redo log'.

NTFS Change Journal implements a circular logging mechanism and hence it is of finite size. NetBackup Accelerator uses it in conjunction with track log. The track log keeps track of the status of records in NTFS change journal instance so that even if the change journal rolls over track log can detect changed files and send the changed segments. Thus, NetBackup Accelerator not only lets you take advantage of NTFS change journal, but adds more value to its usability. 

[AbdulRasheed]

"Does anyone know if the tracking folder and the change journal data are purged after successful backups?". Sorry I hit the wrong 'reply' button!

[AbdulRasheed]

Historically TIR plus move detection was a requirement to use Optimized Synthetic backups. NetBackup Accelerator eliminated that requirement.

However NetBackup Accelerator does support TIR if the business requirement is to have TIR and move detection. Hence you get best of both worlds with NetBackup Accelerator (the flexibility to turn on or off TIR as you need it). 

TIR and NTFS change journal do not go well togather. This is not an Accelerator limitation. NetBackup Accelerator will ignore NTFS change journal automatically if TIR is turned on as discovered by Stanleyj. 

Now the decision is.... do I want TIR for the value it brings during restores or do I need change journal for the performance boost it can provide during backups? The intelligent default in Accelerator is to favor TIR becuase the user had made a deliberate decision to turn it on (TIR is off by default in policies unless you turn on BMR). The good news is that even without NTFS change journal, the built-in track log in Accelerator speeds up backups as you have already found! 

[AbdulRasheed]

You are absolutely right! But I want to clarify that the 'seeding advantage' you are getting here is not from Accelerator, but it is coming from NetBackup Client Side Deduplication. See my blog when you get a chance. This is one reason why I love the combination of using NetBackup Accelerator with NetBackup Client Side Deduplication. 

By the way, when new clients are added, you have an option to 'pre-seed' by telling NetBackup that the client in question have similar data as a pre-exisiting client. The new client will pre-load the fingerprints from the pre-exisiting client's recent backup for deduplication. That way, even the very first backup of a brand new client will run quite fast as it already 'knows' about the fingerprints available in the storage. Check the NetBackup Deduplication Guide for 7.5 release! (hint: search for FP_CACHE_CLIENT_POLICY )