cancel
Showing results for 
Search instead for 
Did you mean: 

question on limit i/o streams per volume

WVT
Level 4
Partner Accredited Certified
When you enable the Maximum I/O streams per volume option on the Disk Pool the default number of streams is 2. Although the optimal number of streams per volume will vary depending on the disk type, a general guide line to minimize contention would be to divide the maximum concurrent jobs count for all the storage units using the disk pool by the number of volumes in the disk pool. This snippet from NBU guide would seem to indicate that if I have a single storage unit using a single MSDP pool with , let's say 5 underlying volumes, and there are no jobs but backup jobs, I would set max concurrent jobs to 100 and the per volume stream limit to 20, correct? NBU uses all volumes in a pool in parallel , providing there are enough jobs, correct?
8 REPLIES 8

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

MSDP pool uses 1 volume only. 
You cannot create an MSDP pool across multiple volumes/luns.

If hardware array is used with striping/Raid across 5 disks, the OS and NBU has no knowledge of this. 
The OS and NBU will see 1 volume only.

WVT
Level 4
Partner Accredited Certified
In the case of an appliance, if I look at the pool properties, there are multiple volumes underneath.

Nicolai
Moderator
Moderator
Partner    VIP   

I think the docs and explaining of that option is highly ambiguous

Netbackup doen't know how many LUNs or disk is used for a given storage unit. The value is the maximum number of concurrent read or write stream the storage unit should handle.

In your case 100 and not 20.

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Storage Foundation on an Appliance creates a single volume for dedupe storage. You may see a number of mount points on an Appliance, but only one for Dedupe pool. Even if you have 4 Expansion modules on the Appliance, there is still only one volume in the dedupe pool.

Nicolai
Moderator
Moderator
Partner    VIP   

Yes - in other words - Netbackup see a file system. Not how its made

WVT
Level 4
Partner Accredited Certified
Perhaps there is some confusion. I am not talking about OS entities. If I right click any Openstorage disk pool in NBU, there may be one or more, what NBU calls 'disk volumes' that comprise the pool.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
For MSDP there can only be one volume in the disk pool. Advanced Disk can have more than one volume in the disk pool.

sdo
Moderator
Moderator
Partner    VIP    Certified

No matter how the disk storage is comprised - the max achievable sustained IO levels under load... all depend upon so many different factors.  It is perhaps easier to baseline, benchmark and monitor IO levels on de-dupe storage that is private to a NetBackup Media Server - but even then one needs to grasp all of the abstraction layers all the way from OSI level 1 (physical/electrical) up to the application layer.  Not easy.  One has to understand all of the IO abstractions at each layer - i.e. IO in and IO out - it comes from somewhere and goes to somewhere, in some form.  Needless to say, this is much much harder if a shared storage platform, or a shared storage network, is being used.  I think this demonstrates why no-one can ever give a hard and fast number.

Consider this... the IO packet sizes are different as the data moves up and back down through the layers, each time being chopped up and/or transformed in some way:

spindles -> disk blocks -> volume-manager/file-system -> bpbkar -> TCP -> bptm -> bpdm -> de-dupe/OST -> file-system/volume-manager -> LUN/SCSI/iSCSI -> SAS/FC/TCP -> RAID -> stripe -> spindles -> disk blocks

Finally, the efficiency, performance and reponsiveness of de-dupe is also utterly dependent upon the nature of the data being backed-up - which only you have visibility of.

In a nut-shell, it could be construed as mis-leading if any of us were to give you any hard and fast guidelines. So, the usual advice will have to suffice... i.e. initially be conservative, establish several metrics that you can measure... and slowly increase the max allowed job counts... and try to identify the peformance cliff, i.e. the point at which either one or more of "performance or efficiency or responsiveness or capacity or throughput" takes a nose dive.