First of all let me explain you our Backup infraestructure.
We've got a Netbackup 5230 appliance (22.214.171.124) with 25TB cabine in the main site. Copies in the same site are working with no issues. Now, we've got a remote site with a VMWARE ESX 5.1. This server has a file server with Windows 2012 (fileserver) and vmware backup host with Windows 2008 R2 (BHserver), both servers connected to the same datastore in a HP SAS cabine connected to the server. Sites are connected by one 100 Mb/s point to point circuit.
The issue is it seems deduplication isn't being done in the client side but in Appliance. We've got set in fileserver "Always use client-side deduplication" and in the job policy BHserver is selected as VMware backup host.
I'm testing with boot disk (40GB). First copy took 77 minutes. (log1.txt)
We expected second copy was faster because the data sent should be fewer, but it took the same time. (Log2.txt)
As you can see were sent 444151 KB and a 98.9% of dedup theorically. How is it possible it's taking the same time? I was checking network with Traffic 10 0 and it was sending to 80Mb all the time. (and 40765163KB x 8 / 1024 / 80 / 60 = 66 minutes, near to 73 minutes it took.) Because of this I think dedup is done in target side not source.
The rest of the tries are taking the same time aprox (log3.txt)
How can we are sure or enforce client side deduplication?
Many thanks for your help.
Solved! Go to Solution.
now its time to review your lincences ..
as Marianne said, ... you did list the dedup license in your previous post where you listed your lincense options..
since its same lincencse for dedup and Accelerator, and you are saying that you are not able to select the Accelerator option.. it time to review your licenses and go for it as Marianne said...
The backup host (BHserver) at site B is not an MSDP media server, and so all data is being sent from VM backups to site A. To realise client side dedupe of the VMs at site B you will need to backup the VMs using a NetBackup Client inside the VMs.
Alternatively, if you have a capacity based license, you could make the BHserver an MSDP media server, and then backup the VMs locally at site B, and then use SLP and Opt-Dup (Optimised Duplication) to duplicate the backups from BHserver at site B to your appliance at site A - which would result in reduced WAN traffic.
does the backup host also the Media server or Media server is located in Main site?
did you try the accelerator feature with Vmware backup type.. i see it is giving the fater backups
the logs that you attached everything is showing the time between "23/09/2015 13:48:27" to "23/09/2015 14:01:26"
looks like they are not complete logs..
Right, Media server it's in Main site, it's the Master server too as it's an appliance.
RamNagalla, I tried Accelerator with Free license trail, it's awesome (as the price for my budget :( )
Sdo, when we talk about Netbackup client inside VMs it's related to install client software in BHserver, right? It's done both BHserver and Fileserver.
Sorry, I uploaded non the right ones. Attaching logs again (one by one, it seems there is a bug in webpage or my browser)
If you use client side de-dupe from a NetBackup Client inside all VMs then you won't have need for the BHserver. But then any VM recovery will either be a BMR based approach, or better still a recover to VM - which should make things simpler.
Sorry, I think I've got a mess and a lack of knowledge.
So, if my fileserver (which is a vmachine in a stand alone ESX) has installed NB client I don't need BHserver (vmachine in the same ESX), so do I have to select as VMware backup host "Backup Media Server" which is in the main site?
Doing this the copy takes more time (from 80 Mb/s to 50Mb/s) misteriously. I was looking into it with a Symantec support and can't explain me why. But this is other matter.
Coming back to dedup, B site is a small one, I just need be able to enable client side dedup. I can restore full machine without BMR, do you think I need one only for one machine? Can't I avoid to install a BMR server to de-dup?
I think several of the Blueprints for NetBackup may help you better understand your options:
Re your questions:
1) If you use plain client backups, then a "NetBackup VM Backup Host" is not involved in backups or restores. I.e. the backup is neither SAN nor NBD based, it will be plain client based.
2) I would expect a plain client backup to be slower because the NetBackup Client software inside the VM is having to walk the file systems. Whereas your current backup is probably NBD transport based (i.e. the ESX server is reading the local LUNs (data stores)) directly at the block level and so should be quicker.
3) To perform a 'create a VM from a plain backup' requires BMR. You shoudl easily be able to configure BMR on your master server NetBackup appliance. You will have very few BMR backups, and so the BMR database side of things will not be very big at all. Remember to enable BMR and TIR on the backup policy which will backup the file server VM.
P.S: Please do keep coming back with more questions. We'll try to help :)
did you enable this option in vmware policy type, Enable file recovery from VM backup.
if yes.. i am only seeing enabling with accelerator feature is the best solution and if you have capacity based licence it does not cost you.
unless you want to go with BMR as suggested by SDO, no more Vmware policy type backups.. its going to be Ms- windows policy type with BMR enabled.
Good point Ram - @Angel, you may be able to reduce the volume of data picked-up by the VM Backup Host BHserver during any full or cumulative or differential VM style backup... by selecting 'Use Accelerator' on the VM based backup policy. This may be enough to reduce your WAN traffic without having to look at the options that I've previously covered - but should also vastly reduce the backup duration (following the first initial full backup using accelerator) - depending of course upon the rate of data change within the virtualised file server.
Many thanks for your help. I'm going to take a look into blueprints ;)
BTW, taking advantage of this (and I promise you I read all the documentation I found :) ) the other problem I have with this policy is Wan Optimization. Although I've got set it up, all the traffic is sent as "Non Optimized"
Network Optimization = ENABLED
TCP connections not optimized (or not appropriate for optimization) = 0
TCP connections optimized = 3
Attached you can see a screen shot of traffic when only this policy is working.
Even I created a Resilent Network between Master server and fileserver with the same result.
Do I have to do anything else?
Yep, "enable file recovery" is enabled.
Unfortunately it seems I don't have capacity based licence. I've just reviewed them and this is that I've got. Cheking the appliance webpage administrator it seems not be available :(
|SYMC NETBACKUP 5230 APPLIANCE 4TB SUPPORT NBD ONSITE RENEWAL 12 MONTHS|
|SYMC NETBACKUP SAP AGENT 7.6 WIN/LNX/SOLX64 1 SERVER TIER 2 RENEWAL BASIC 12 MONTHS EXPRESS BAND S|
|SYMC NETBACKUP ENTERPRISE SERVER 7.6 WIN/LNX/SOLX64 1 SERVER TIER 2 RENEWAL BASIC 12 MONTHS EXPRESS BAND S|
|SYMC NETBACKUP ENTERPRISE CLIENT 7.6 WIN/LNX/SOLX64 1 SERVER TIER 2 RENEWAL BASIC 12 MONTHS EXPRESS BAND S|
|SYMC NETBACKUP CLIENT APPLICATION AND DATABASE PACK 7.6 WIN/LNX/SOLX64 1 SERVER TIER 2 RENEWAL BASIC 12 MONTHS EXPRESS BAND S|
|SYMC NETBACKUP 52X0 APPLIANCE 24TB DUAL CONTROLLER STORAGE SHELF ESSENTIAL SUPPORT INITIAL 12 MONTHS|
|SYMC NETBACKUP 52X0 APPLIANCE EXTERNAL STORAGE KIT PHYSICAL INSTALLATION SERVICE ONLY ONE TIME FEE|
Have you tried enabling 'Use Accelerator' on the backup policy?
FYI - after enabling accelerator, the benefit of accelerator will only be seen from the second backup onwards.
Unfortunately I don't have licence for it so the check is disabled and I don't have budget for it.
I think if I get doing de-dup in the source client won't need Accelerator at the moment and will be able to copy fileserver in a reasonable time
please apologize for overlooking your first post
"The issue is it seems deduplication isn't being done in the client side but in Appliance. We've got set in fileserver "Always use client-side deduplication" and in the job policy BHserver is selected as VMware backup host"
you have selected "Always use client-side deduplication" for file server, but for Vmware policy backup host act as a client.
in your case BHserver is the one that you need to set, please set Always use client-side deduplication for BHserver and try the backups again, and let me know if how goes..
No problem Ram, most of all thanks for all your time, guys.
About the issue, I've got set client-side dedup in BHserver. The full backup is just ended and it took more than 48H for 1,45TB, so it's sending all the data without dedup again :(
Only to be sure my concept of client-side dedup is right, my idea is with this option set up, the VM backup Host should find out what info/blocks aren't in the MSDP (in Main Site) and then only send them by the WAN. So, in this case:
scanned: 40765163 KB, CR sent: 444151 KB, CR sent over FC: 0 KB, dedup: 98.9%, cache hits: 354068 (97.6%), rebased: 2214 (0.6%)
BHserver should send only 444151KB (CR sent), right?