cancel
Showing results for 
Search instead for 
Did you mean: 

client side dedup issue

Angel_Fontana
Level 4

Hi there,

First of all let me explain you our Backup infraestructure.

We've got a Netbackup 5230 appliance (2.6.1.1) with 25TB cabine in the main site. Copies in the same site are working with no issues. Now, we've got a remote site with a VMWARE ESX 5.1. This server has a file server with Windows 2012 (fileserver) and vmware backup host with Windows 2008 R2 (BHserver), both servers connected to the same datastore in a HP SAS cabine connected to the server. Sites are connected by one 100 Mb/s point to point circuit.

The issue is it seems deduplication isn't being done in the client side but in Appliance. We've got set in fileserver "Always use client-side deduplication" and in the job policy BHserver is selected as VMware backup host.

I'm testing with boot disk (40GB). First copy took 77 minutes. (log1.txt)

We expected second copy was faster because the data sent should be fewer, but it took the same time. (Log2.txt)

As you can see were sent 444151 KB and a 98.9% of dedup theorically. How is it possible it's taking the same time? I was checking network with Traffic 10 0 and it was sending to 80Mb all the time. (and 40765163KB x 8 / 1024 / 80 / 60 = 66 minutes, near to 73 minutes it took.) Because of this I think dedup is done in target side not source.

The rest of the tries are taking the same time aprox (log3.txt)

How can we are sure or enforce client side deduplication?

Many thanks for your help.

Regards.

 
24 REPLIES 24

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

hi Angel,

the above listed licences are not showing the dedup licences unless you miss to post it though you already have..

and could you post the all detail status of the job the is compleated...? does it the first job after enabling the client side dedup for BH server?

if it really trying to use the client side dedup.. in job detail status there should be a line some thing like below.

 Using OpenStorage client direct to backup from client xyz to abc

if you dont find the line similer to this in job details status.. it means that is it not using the client side dedup.. and needs to confim about the licences or configurations again.

if you see the lines simler to this in job detail status.. it is using the client side dedup and CR send is the only one that is send from the client.

 

Angel_Fontana
Level 4

Here again,

This is the log for the full backup (it wasn't the first)

02/10/2015 16:31:26 - Info nbjm(pid=17290) starting backup job (jobid=47855) for client omgbesfiles.global.com, policy VM_OMGBESFILES, schedule Completa  
02/10/2015 16:31:27 - estimated 0 Kbytes needed
02/10/2015 16:31:27 - Info nbjm(pid=17290) started backup (backupid=omgbesfiles.global.com_1443796284) job for client omgbesfiles.global.com, policy VM_OMGBESFILES, schedule Completa on storage unit stu_disk_omgesnetbu using backup host OMGBESAPPS.global.com
02/10/2015 16:31:27 - started process bpbrm (73536)
02/10/2015 16:31:28 - Info bpbrm(pid=73536) omgbesfiles.global.com is the host to backup data from     
02/10/2015 16:31:28 - Info bpbrm(pid=73536) reading file list for client        
02/10/2015 16:31:28 - connecting
02/10/2015 16:31:29 - Info bpbrm(pid=73536) starting bpbkar on client         
02/10/2015 16:31:29 - connected; connect time: 0:00:01
02/10/2015 16:31:30 - Info bpbkar(pid=712) Backup started           
02/10/2015 16:31:30 - Info bpbrm(pid=73536) bptm pid: 73543          
02/10/2015 16:31:31 - Info bptm(pid=73543) start            
02/10/2015 16:31:31 - Info bptm(pid=73543) using 262144 data buffer size        
02/10/2015 16:31:31 - Info bptm(pid=73543) using 30 data buffers         
02/10/2015 16:31:31 - Info bpbkar(pid=712) archive bit processing:<enabled>          
02/10/2015 16:31:31 - Info bptm(pid=73543) start backup           
02/10/2015 16:31:37 - Info bptm(pid=73543) backup child process is pid 73596       
02/10/2015 16:31:37 - begin writing
04/10/2015 19:29:59 - Info bptm(pid=73543) waited for full buffer 5819344 times, delayed 12019792 times    
04/10/2015 19:30:06 - Info bptm(pid=73543) EXITING with status 0 <----------        
04/10/2015 19:30:06 - Info omgesnetbu(pid=73543) StorageServer=PureDisk:omgesnetbu; Report=PDDO Stats (multi-threaded stream used) for (omgesnetbu): scanned: 1490495656 KB, CR sent: 9239012 KB, CR sent over FC: 0 KB, dedup: 99.4%, cache hits: 827941 (6.7%), rebased: 163899 (1.3%)
04/10/2015 19:30:07 - Info bpbrm(pid=73536) validating image for client omgbesfiles.global.com        
04/10/2015 19:30:07 - Info bpbkar(pid=712) done. status: 0: the requested operation was successfully completed    
04/10/2015 19:30:07 - end writing; write time: 50:58:30
the requested operation was successfully completed (0)

 

As you can see there is nothing related with OpenStorage client direct. So, would it be that we are able to dedupe data in MSDP (target dedupe) but not to dedupe in source because we need that license only for client-side deduplication?

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
There is not a separate Accelerator license. Dedupe and Accelerator are included in Data Protection Optimization license. You did not mention this license in your post where you quoted licensed options?

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

now its time to review your lincences ..

as Marianne said, ... you did list the dedup license in your previous post where you listed your lincense options..

since its same lincencse for dedup and Accelerator, and you are saying that you are not able to select the Accelerator option.. it time to review your licenses and go for it as Marianne said...

Angel_Fontana
Level 4

Hi guys,

Ok, it takes me long time to understand the issue. When I was talking with my vendor always we thought in dedupe option. The budget was adjusted and I guess the vendor cut back this license. While we didn't have to backup remote site didn't have any issue. Our MDSP was deduping data in its storage, and this data was copied by LAN & SAN, so there was enough time to do it.

Furthermore, I saw

"scanned: 1490495656 KB, CR sent: 9239012 KB, CR sent over FC: 0 KB, dedup: 99.4%, cache hits: 827941 (6.7%), rebased: 163899 (1.3%)", 

client-side option available and I guessed dedupe was working (and licensed).

Well, I try to buy it next year if the budget allows me. At the moment I handle it without Acelerator. I have to optimize my WAN acelerator but this will be another post :D

Thanks a million @Ram, @Marianne & @sdo for all your support and time.

Regards.