Best Setting for max concurrent write drives, max Streams per drive, limit jobs per policy and media multiplexing

[imashu]

Hi,

I have a small environment with only 2 drives attached with the master cum media server on a linux platform with Netbackup 7.5.

Need to know the best setting for the below so as to have efficient usage of my resources;

1) Max concurrent write drives in Storage unit

2) Max Streams per drive in Storage unit

3) Limit jobs per policy in Policy

4) Media Multiplexing in Policy >> Schedule

Also, please update me if i need to follow some other steps as only 2 streams run at a time and backups tend to take very very long time to complete.

[StefanosM]

your ideal is not to use multiplexing - this is what you have now. It is good for the restore.

For most installations, this can not be accomplished, because there are many data to backup and the backup time gets bigger than the backup window.
 

So your goal is to use the lowest possible multiplexing value, to meet the backup window requirement.

(I do not  understand if you have one master/media server or one master and one media server)

What I'm doing with small environments is:

Max concurrent write drives in Storage unit = max no of drives
Max Streams per drive in Storage unit =  20
Limit jobs per policy in Policy = unchecked (but I can use it if I need to limit the specific policy)
max jobs per client = 10 (default 1)
enable multiplexing in policy attributes = checked

and play only with Media Multiplexing in Policy

I'm starting with a number of 7 and if I'm in the backup window I decrease it to 6 and so on. The lower this number can go, the better.

[Marianne]

We find excellent performance with multiplexing set to 4 and still getting good restore performance.

1) Max concurrent write drives in Storage unit
All the drives you have: 2

2) Max Streams per drive in Storage unit
4 is a good trade-off between backup and restore performance

3) Limit jobs per policy in Policy
we leave it unlimited, unless one particular policy generate so many streams that it occupies all available drives and streams.

4) Media Multiplexing in Policy >> Schedule
Best to match MPX in STU config. 
The lowest of the 2 settings get applied when backup is started and becomes limiting factor.

Good practice is to also reduce the fragment size in the STU (5000).
This ensures faster restores.

Best is to start slowly, change MPX settings in STU and schedules to 2.
Check results. If good, change to 3, and so on.

Do regular restore tests.

Hope this helps.

[imashu]

Hi Marianne,

I have done the changes as mentioned by you.

1) Max concurrent write drives in Storage unit - 2

2) Max Streams per drive in Storage unit - 2

3) Limit jobs per policy in Policy - Unchecked

4) Media Multiplexing in Policy >> Schedule - 4

My fragment size is 1048575 megabytes. What should be the best size for backup and restores.

What impact does it have?

[Marianne]

"4) Media Multiplexing in Policy >> Schedule - 4 " will have no effect.  You will get 2 streams per drive because that is the MPX setting in the STU.

As per my previous post - the lower number of the 2 MPX settings will be applied. 
Best to keep these two settings the same.

We get good backup and restore performance with fragment set to 5000. 

[StefanosM]

For LTO5 I use fragment size 20000 to 25000, The same I use for disk backups.

And always test your backup and restore speed after every change.

[imashu]

Thanks Marianne,

After doing the recommended changes we are still seeing that only 1 drive is being used for backup's.

There are 2 streams for 1 policy and only 1 stream runs on only 1 drive and same with catalog backup's as well..It doesn't becomes active if any other backup is running. Although sometimes it runs fine for other policies.

I have already removed mds allocation and done reset all twice but still no luck.

Kindly suggest!!

[Marianne · Accepted solution]

We need to see your config, please.

Post output of: 

bpstulist -U

bppllist <problem-policy> -U

Please also check what is Max Jobs per Client is set to in Host Properties -> Master -> Global attributes.

One more thing: 
Please check that the Volume Pool specified in the policy has enough available media.

Do you see jobs queued when only one drive is in use?
If so, what reason is listed in Details tab?

[imashu]

Hello Marianne,

I think i got where the problem was.. Max jobs per client was set to 1 in Host Properties -> Master -> Global attributes. I have now made it 4. Since this policy and catalog uses the same server so they remain queued wheras other client runs fine.

Have run the "bprdreq -rereadconfig" . It seems i will have to restart the services.

Please find the required output;

/usr/openv/netbackup/bin/admincmd bpstulist -U

[root@buk003bkup001p admincmd]# ./bpstulist -U

Label:                BUK003BKUP001P-hcart2-robot-tld-0
Storage Unit Type:    Media Manager
Host Connection:      BUK003BKUP001P
Number of Drives:     2
On Demand Only:       no
Max MPX/drive:        4
Density:              hcart2 - 1/2 Inch Cartridge 2
Robot Type/Number:    TLD / 0
Max Fragment Size:    1048575 MB

bppllist AG_BO_ProdUK -U

[root@buk003bkup001p admincmd]# ./bppllist AG_BO_ProdUK -U
------------------------------------------------------------

Policy Name:       AG_BO_ProdUK

  Policy Type:         Standard
  Active:              yes
  Effective date:      03/14/2013 09:18:58
  Client Compress:     no
  Follow NFS Mounts:   yes
  Cross Mount Points:  yes
  Collect TIR info:    no
  Block Incremental:   no
  Mult. Data Streams:  yes
  Client Encrypt:      no
  Checkpoint:          yes
       Interval:       15
  Policy Priority:     99999
  Max Jobs/Policy:     Unlimited
  Disaster Recovery:   0
  Collect BMR info:    no
  Residence:           BUK003BKUP001P-hcart2-robot-tld-0
  Volume Pool:         DataStore
  Server Group:        *ANY*
  Keyword:             (none specified)
  Data Classification:       -
  Residence is Storage Lifecycle Policy:    no
  Application Discovery:      no
  Discovery Lifetime:      0 seconds
ASC Application and attributes: (none defined)

  Granular Restore Info:  no
  Ignore Client Direct:  no
Enable Metadata Indexing:  no
Index server name:  NULL
  Use Accelerator:  no
  HW/OS/Client:  Linux         RedHat2.6.18  BUK003BKUP001P

  Include:  /ebsukprddata
            /ebsukprdarch

  Schedule:              Yearly_Full
    Type:                Full Backup
    Frequency:           every 360 days
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     9 (infinity)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:

  Schedule:              Monthly_Full
    Type:                Full Backup
    Frequency:           every 21 days
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     8 (1 year)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:

  Schedule:              Weekly_Full
    Type:                Full Backup
    Frequency:           every 5 days
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     4 (2 months)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:

  Schedule:              Daily_Full
    Type:                Full Backup
    Frequency:           every 18 hours
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     1 (2 weeks)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:

[root@buk003bkup001p admincmd]#

[Marianne]

Glad I could point you in the right direction!

All should be fine as soon as NBU on master has been restarted.

STU and schedules all look fine with MPX set to 4.

[imashu]

Hello Marianne,

Yes it's now resolved and i have set max. jobs per client to 8 and many jobs are running simultaneously now.

Many Thanks!!