Deduplication Bandwidth limiting

[rbkguy]

Hey everyone!

I am looking for some info on how to limit the bandwidth\traffic coming into my deduplication media server. My current setup is:

Windows 2008 R2 Master running 7.5.0.4

Windows 2008 R2 Media running 7.5.0.4

(5) regional offices with 4 servers at each site with a 10MB wan link to a cloud and then a larger 25MB pipe into head office where the media server exist. 

I have close to 25 servers that I have been testing client side deduplication and it works great except that I choke the head office pipe. I am able to limit the bandwidth fine coming from the server but I would like to limit what the deduplication media server recieves. Is this something that can be done?

[RamNagalla]

hey, 

its looks like you can achive it by using bandwidthlimit in agent.cfg

see the below that i got from Deduplication admin guide page 89

About configuring optimized duplication and

replication bandwidth

Each optimized duplication or Auto Image Replication job is a separate process

or stream. The number of duplication or replication jobs that run concurrently

determines the number of jobs that contend for bandwidth. You can control how

much network bandwidth that optimized duplication and Auto Image Replication

jobs consume.

Two different configuration file settings control the bandwidth that is used, as

follows:

bandwidthlimit

The bandwidthlimit parameter in the agent.cfg file is the

global bandwidth setting. You can use this parameter to limit the

bandwidth that all replication jobs use.

If bandwidthlimit is greater than zero, all of the jobs share the

bandwidth. That is, the bandwidth for each job is the

bandwidthlimit divided by the number of jobs.

If bandwidthlimit=0, total bandwidth is not limited. However,

you can limit the bandwidth that each job uses. See the following

OPTDUP_BANDWIDTH description

.

By default, bandwidthlimit=0.

OPTDUP_BANDWIDTH:-

The OPTDUP_BANDWIDTH parameter in the pd.conf file specifies

the per job bandwidth.

OPTDUP_BANDWIDTH applies only if the bandwidthlimit

parameter in the agent.cfg file is zero.

If OPTDUP_BANDWIDTH and bandwidthlimit are both 0,

bandwidth per replication job is not limited.

By default, OPTDUP_BANDWIDTH = 0.

OPTDUP_BANDWIDTH

To specify the bandwidth, edit the configuration files on the deduplication storage

server, as follows:

■ bandwidthlimit.

The agent.cfg file resides in the following directory:

■ UNIX: storage_path/etc/puredisk

■ Windows: storage_path\etc\puredisk

■ OPTDUP_BANDWIDTH.

See “About the pd.conf configuration file for NetBackup deduplication”

on page 119.

See “Editing the pd.conf deduplication file” on page 126

Hope this helps.

[Mark_Solutions]

I am not sure that the opt_dup or bandwidth files will help you as this is backup data rather than duplication / replication data - assuming your remote servers are using client side de-dupe rather than duplicating or replicating from a remote office media server back to a central media server

You may be better to use the bandwidth or throttle bandwidth setings in the Master Servers Host properties to control each host / IP range for each office

Hope this helps

[rbkguy]

Is there anyway I can limit the amount of data coming into my HQ media server like 17Mbps and what is coming out of a remote media server? I need to limit my bandwidth to 7Mbps coming from my remote sites but also limit what the head office is excepting.

[rbkguy]

I have been using a local deduplication media server and working with opt_dup and bandwitch files but I am only get one of the two working. I have no problems limiting what comes out of the remote sites but when I have 5 sites sending more then what the pipe can handle in HQ I have a problem.

[Mark_Solutions · Accepted solution]

The OptDup Bandwidth limit set in the pd.conf is per server.

If you have just one media server on each site then it will limit that one but not the global amount coming into your HQ

The bandwidthlimit setting is supposed to be the global one - to be safe set this on the Master and all Media servers to prevent the total data being passed exceeding your allow limit.

This is where the fun begins - what to set it to and what does the value represent?

Well .. it should be a value in KB/s so if set to 2000 it should only allow 2MB/s to pass - but it doesnt quite seem to work that way!

The best i have come up with by trial and error is to think of the number you want in MB/s and add two zeros to it.

Due to your setup (the 10MB and 25 MB pipes maybe try the following, assuming you only want to use half of your bandwidth:

All remote servers use pd.conf setting of 500 (5MB/s plus two zeros) and then everything at the HQ uses the agent,cfg with a setting of 1200 (12 MB/s plus two zero)

If the agent.cfg overrides the pd.confs then your remote sites will still use 1200 which will be too much so you may have to drop it all down - again trial and error may be the way to work it out

Hope this helps