Limits

[cgoliver]

Hello,

We are trying to decode the job limit for storage unit groups and suspect that it's a simple linear relationship

# Media servers * # of concurrent drives * # tape drive types

For our case this is 14 * 2 * 3 = 84

But, we are not sure.

[Nathan_Kippen]

So... What I do is for every tape drive STU I set the max concurrent jobs per drive to 4. Then in my policies I set my multiplexing to 4 jobs per stream in my schedules.. that way I am writing 4 backup streams per tape drive. Then you can decide how many jobs per policy you want to run at the same time.

[cgoliver]

We've set no limit on the policy, so it can potentially run hundreds of jobs.

We're more interested in whether the equation suggested is accurate.

[Nathan_Kippen]

You set limits on each storage unit, policy, and schedule..  There are no limits set on a storage unit groups.

From experience you really don't want more than 4 jobs writing to a single tape drive at the same time.

So if you have 10 tape drives.. the most simultaneous jobs writing at once could be 40.

[J_H_Is_gone]

does not matter how many media servers you have only 1 media server can write to a tape drive at a time.

Thou each media server can send more than 1 job to that tape drive

so #tape drives * the number of max writes set (the lower number of - schedule or storage unix settings)

[cgoliver]

I've been removing all job limits, but have kept the concurrent drive count low. We still have jobs queue

even though avaiable resources sit empty.

My concern is that if I set the concurrent drives to equal the number of drives available for that type, then a single media server will monoplise all resources. The goal is to maximize parellelism as we have about 37 tapes drives and 18 media servers, which backup ~ 300 clients per day

Fulls are about 80 TB, differential incr is about 12 TB

Thanks,

Chris

[J_H_Is_gone]

its all based on how long do jobs run on media server 1  - 8 hours  - then it might not give up drives

media server 2 might only run 2 hours of your 8 hour window so 1 drive and then that 1 drive could be used by a different server.

but then the trick comes in with priority and start time  maybe start media server 2 first so it gets it drive then start media server 1's backups and it starts with only 4 of 5 drives but then when media server 2 is done media server 1 can grab it.

but with so many media servers you really have to to play with how many drives in a storage unit and multiplex and streams per policy

you could see that you give it 5 drives but only allow so many jobs per policy - and the multiplex is so high that it only takes 3 of the 5 drives....

one of those moving targets - that you just keep tweaking.

[Mark_Solutions · Accepted solution]

Here is what i have done recently for a customer to ensure everything kicks in efficiently

Looking at your origial equation suggests you have different drive type and hence each Media server has more than one storage unit to put in the group.

So if we have for simplicity 3 media servers (M1, M2 and M3) each having 2 storage units (M1-S1, M1-S2, M2-S1 etc.)

then the group would be set up as round robin and the storage units in that group would be adding in the correct order (remember you can "move up" and "move down" in a group:

M1-S1

M2-S1

M3-S1

M1-S2

M2-S2

M3-S2

In this way ot ensure that it just works its way around then all in order and balances things out

Hope this helps

[cgoliver]

With some many tapes and tape drives (3000), (37) the potential for problems is greater with round-robin, but I've decided to give media server load balancing a try.

What we have setup today is

M1-S1

M1-S2

M1-S3

M1-S4

M2-S1

and so on.

This kinda goes back to an earlier question. We were told at one point that this should be limited due to the job management overhead in nbjm and associated processes.

Due to the number of tape drives and media servers, there are 32 storage units in my Production STUG.

Thanks,

Chris