cancel
Showing results for 
Search instead for 
Did you mean: 

Host requirements for client-side-deduplication

jmarques68
Level 3
Partner Accredited

I don’t know if someone can help me on getting at least an approximate idea of the CPU/RAM requirements for the clients that use client-side-dedup in an NetBackup environment, do you have any metrics that can reflect an estimation of those requirements? For example, if a server that is running exchange and as 2 CPUs, 8GB of RAM and data volume equal to 2TB for backup, what should be the impact of enabling client-side-dedup on the existing resources?

1 ACCEPTED SOLUTION

Accepted Solutions

Jaime_Vazquez
Level 6
Employee

Having just had a small class that somewhat covered this, this is what I could infer:

For the client: At least a dual core 64 bit processor >= 2.5 GHz. Naturally, more is better.

RAM depends on the backup selection size. There is supposed to be documentation concerning the requirements for this.I would think a minimum baseline is 4 GB, but again, more is better.

During the client side dedupe process lots of shared memory is used on the client to hold the SO's associated with the backup being performed. The larger the backup to be done, the more memory needed to cache the SO entries of its own existing backups. It is also fairly compute intensive as it scans the files, creates the associated SO fingerprints, and checks the SO 'fingerprints' it creates from the PO selections it is running against the cached memory SO entries it is holding. The amount of RAM needed increases as the number of SO entries grows.

The mechanism of client-side deduplication is a means by which each client can 'filter out' those SO entries that it knows about that are already in existence on the storage server. Instead of sending out all of its generated SO fingerprints to the server, it only sends out what it see as new SO fingerprint entries to it.  The subset of entries is then checked out in the cached memory of the server itself to see if it already holds the same SO entries, due to backups for other clients. 

Client side dedupe has no effect on the needed RAM requirements of the Server, as it must always hold all of the active SO entries for all of its PO/DO entries. Client side dedupe reduces the amount of data transferred from client to server and reduces the workload needed to process that data on the Server. I would think it a hard metric to determine how much work is being offloaded from the server back to the client for this. .As I would say - "Your mileage will vary, subject to change without notice, void where prohibited, must be over 18 to play".  8-)

 

SO – Segment Object

PO – Path Object

DO – Data Object

 

View solution in original post

2 REPLIES 2

symantec040
Level 4

Hi,

RAM -> For every 1TB of dedupe data should have 1GB of RAM installed. You can continue to scale till the software allows you to a level.

CPU -> from their 5220 appliance,

Architecture:          x86_64
CPU(s):                16
Thread(s) per core:    2
Core(s) per socket:    4
CPU socket(s):         2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 44
Stepping:              2
CPU MHz:               1596.000
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              12288K

It is good to have 64bit, dual core >2.0GHZ in general.

 

The above metrics apply to the media server too. To be more specific on the client dedupe - please be sure, the client still has breathing space after certain amount of memory is occupied by Netbackup processes. This is something your OS/Application teams can tell - what amount of memory would their applicaitons require.

Jaime_Vazquez
Level 6
Employee

Having just had a small class that somewhat covered this, this is what I could infer:

For the client: At least a dual core 64 bit processor >= 2.5 GHz. Naturally, more is better.

RAM depends on the backup selection size. There is supposed to be documentation concerning the requirements for this.I would think a minimum baseline is 4 GB, but again, more is better.

During the client side dedupe process lots of shared memory is used on the client to hold the SO's associated with the backup being performed. The larger the backup to be done, the more memory needed to cache the SO entries of its own existing backups. It is also fairly compute intensive as it scans the files, creates the associated SO fingerprints, and checks the SO 'fingerprints' it creates from the PO selections it is running against the cached memory SO entries it is holding. The amount of RAM needed increases as the number of SO entries grows.

The mechanism of client-side deduplication is a means by which each client can 'filter out' those SO entries that it knows about that are already in existence on the storage server. Instead of sending out all of its generated SO fingerprints to the server, it only sends out what it see as new SO fingerprint entries to it.  The subset of entries is then checked out in the cached memory of the server itself to see if it already holds the same SO entries, due to backups for other clients. 

Client side dedupe has no effect on the needed RAM requirements of the Server, as it must always hold all of the active SO entries for all of its PO/DO entries. Client side dedupe reduces the amount of data transferred from client to server and reduces the workload needed to process that data on the Server. I would think it a hard metric to determine how much work is being offloaded from the server back to the client for this. .As I would say - "Your mileage will vary, subject to change without notice, void where prohibited, must be over 18 to play".  8-)

 

SO – Segment Object

PO – Path Object

DO – Data Object