Forum Discussion

Volker_Spies1's avatar
Volker_Spies1
Level 4
11 years ago

Netbackup Client DeDup & Accelerator savings report

Hi Forum, we are testing Client Side DeDup + Accelerator + DeDup Pool for our branch offices right now. and we see results that we can't explain. We have a client backup that states this: ...
Backup and Recovery
NetBackup
Windows Server (2003-2008)
  • RLeon's avatar
    RLeon
    11 years ago
    4/2/2014 10:30:35 PM - Info bpbkar32(pid=4512) accelerator sent 433504768 bytes out of 1071560704 bytes to server, optimization 59.5%

    While on the client, Accelerator sent 433,504,768 bytes (423,344KB) out of 1,071,560,704 bytes (1,046,446KB) to the Client-Side-Dedup plug-in (we're still on the client here). In other words, 423,344KB is the figure before any deduplication.

    As for the reason why Accelerator reported to have found 1,046,446KB instead of 31,457,000KB, this is due to the way Accelerator 'tracks' changes in the file system. Apparently it has a quick and proprietary way of tracking directories/folders that contain changed data. A glimpse of this info could be found on this post by AbdulRasheed.

    The list of directories Accelerator retrieved has a total size of 1,046,446KB. But not every file and its blocks have changed, so only the changed ones are sent. That gives us the figure 423,344KB, which is 40.5%, or, to put it in another way, "optimization 59.5%".

     

     

    4/2/2014 10:30:35 PM - Info bpbkar32(pid=4512) bpbkar waited 1372 times for empty buffer, delayed 37356 times.

    A bpbkar delay in NetBackup is 10ms. So 37356 delays cost around 6 minutes of not being able to send any data.
    In your original post there were 736141 delays. That means at least 2 hours of the 6 hour backup were due to factors outside of NetBackup. You may want to start off by checking the network connection to the branch office. 73GB in 4 hours is around 5.2MB/s, much better than 3MB/s.

     

     

    4/2/2014 10:31:33 PM - Info sv42080oel0197(pid=7896) StorageServer=PureDisk:sv42080oel0197; Report=PDDO Stats for (sv42080oel0197): scanned: 1046466 KB, CR sent: 69031 KB, CR sent over FC: 0 KB, dedup: 93.4%, cache disabled

    Although it is true that Accelerator did initially report 1,046,446KB as we have established previously, but it ended up only having sent 423,344KB to the Client-Side-Dedup plug-in. So the part that says "scanned: 1046466 KB" from the above, could have been 'clearer' by saying something to the effect of: "423344KB of 1046466KB received from Accelerator". But hey.

    So anyway, out of 1,046,466KB, the Client-Side-Dedup plug-in (yep, still on the client here) ended up sending only 69,031KB to the MSDP Media Server. This answers your question of how much data does the Client actually send out.
    69,031KB of 1,046,466KB is around 6.6%, or to spin it another way: "dedup: 93.4%".
    I would have preferred it to say 69,031KB of 423,344KB, which would have made it: "dedup: 83.7%".
    But hey.

     

    As for the TIR header at dedup: 100%, one of the NetBackup training videos on education.symantec.com explains it, verbatim:
    ".. The second number here, it used to just say zero, arh, because the Media Server wasn't doing anything, and then people kinda freaked out: 'Oh no! I'm getting zero percent Dedup!' So they changed it to be a hundred. But either way whether it's a zero or a hundred, it's a meaningless number. You know, the number I wanna look at is the number on the client."

    The name of the video is "Managing NetBackup Deduplication in 5230 Appliances", fyi.

    I'm guessing they changed it from 0% to 100% starting NetBackup 7.5, but didn't really update the documents, release notes or the online technotes to reflect this.

RLeon's avatar
RLeon
Level 6
11 years ago
4/2/2014 10:30:35 PM - Info bpbkar32(pid=4512) accelerator sent 433504768 bytes out of 1071560704 bytes to server, optimization 59.5%

While on the client, Accelerator sent 433,504,768 bytes (423,344KB) out of 1,071,560,704 bytes (1,046,446KB) to the Client-Side-Dedup plug-in (we're still on the client here). In other words, 423,344KB is the figure before any deduplication.

As for the reason why Accelerator reported to have found 1,046,446KB instead of 31,457,000KB, this is due to the way Accelerator 'tracks' changes in the file system. Apparently it has a quick and proprietary way of tracking directories/folders that contain changed data. A glimpse of this info could be found on this post by AbdulRasheed.

The list of directories Accelerator retrieved has a total size of 1,046,446KB. But not every file and its blocks have changed, so only the changed ones are sent. That gives us the figure 423,344KB, which is 40.5%, or, to put it in another way, "optimization 59.5%".

 

 

4/2/2014 10:30:35 PM - Info bpbkar32(pid=4512) bpbkar waited 1372 times for empty buffer, delayed 37356 times.

A bpbkar delay in NetBackup is 10ms. So 37356 delays cost around 6 minutes of not being able to send any data.
In your original post there were 736141 delays. That means at least 2 hours of the 6 hour backup were due to factors outside of NetBackup. You may want to start off by checking the network connection to the branch office. 73GB in 4 hours is around 5.2MB/s, much better than 3MB/s.

 

 

4/2/2014 10:31:33 PM - Info sv42080oel0197(pid=7896) StorageServer=PureDisk:sv42080oel0197; Report=PDDO Stats for (sv42080oel0197): scanned: 1046466 KB, CR sent: 69031 KB, CR sent over FC: 0 KB, dedup: 93.4%, cache disabled

Although it is true that Accelerator did initially report 1,046,446KB as we have established previously, but it ended up only having sent 423,344KB to the Client-Side-Dedup plug-in. So the part that says "scanned: 1046466 KB" from the above, could have been 'clearer' by saying something to the effect of: "423344KB of 1046466KB received from Accelerator". But hey.

So anyway, out of 1,046,466KB, the Client-Side-Dedup plug-in (yep, still on the client here) ended up sending only 69,031KB to the MSDP Media Server. This answers your question of how much data does the Client actually send out.
69,031KB of 1,046,466KB is around 6.6%, or to spin it another way: "dedup: 93.4%".
I would have preferred it to say 69,031KB of 423,344KB, which would have made it: "dedup: 83.7%".
But hey.

 

As for the TIR header at dedup: 100%, one of the NetBackup training videos on education.symantec.com explains it, verbatim:
".. The second number here, it used to just say zero, arh, because the Media Server wasn't doing anything, and then people kinda freaked out: 'Oh no! I'm getting zero percent Dedup!' So they changed it to be a hundred. But either way whether it's a zero or a hundred, it's a meaningless number. You know, the number I wanna look at is the number on the client."

The name of the video is "Managing NetBackup Deduplication in 5230 Appliances", fyi.

I'm guessing they changed it from 0% to 100% starting NetBackup 7.5, but didn't really update the documents, release notes or the online technotes to reflect this.