Forum Discussion

belse's avatar
belse
Level 5
12 years ago
Solved

Multiplexing and Multistreaming

Hi All,

I have some questions about multistreaming and multiplexing.

1:- How many multi stream we can do in policy. I think its 32 and how can we change it?

2:-How many multistream can we do in policy and how can we change it lets suppose we need to stream it 2 gb each?

 

Could somebody tell me how to change the settings for both? The procedure.

 

Thanks,

Belse

7.1.x and Earlier
Backup and Recovery
Basics
NetBackup
Windows Server (2003-2008)
  • Marianne's avatar
    Marianne
    12 years ago

    The theoretical multistreaming limit per client is 99. That is the max value that 'Max Job per Client' in master server Host Properties can be set to.
    The default 'Limit jobs per policy' (policy attribute) is set to unlimited. 

    Factors that determine how many jobs are activated depend on Storage Unit settings - MPX and number of drives as well as MPX settings in Policy Schedules.

    It is never a good idea to go with max settings since clients are hardly ever able to generate more than 4 data streams simultaneously without suffering performance issues.
    Likewise MPX of 32 is never a good idea since media server's ability to write that many streams per device is questionable and greatly affects restore speeds.
    We have found MPX of 4 gives good backup and restore performance.

    Before you chance ANYTHING, please go through these topics in NBU Admin Guide I (in addition to above-mentioned TN): 

    Limit jobs per policy (policy attribute)

    Allow multiple data streams (policy attribute)

    Media multiplexing (schedule attribute)
    About multiplexing

    Example of using multiplexing with schedules

    Once you have read through all of these topics, introduce changes SLOWLY....
    Try Max Jobs per Client, STU MPX and Schedule MPX of 2. 
    Ensure bptm logs exist on media server(s). Review results in bptm log (performance Tuning Guide will tell you what to look fir in bptm logs).
    If performance is good, increase in increments of 2 (4, 6, 8), each time verifying and documenting results.
    At some point clients and/or media server(s) will be overloaded and performance will deteriorate.

    You need to find values that is good for YOUR unique environment.

8 Replies

Sort By
  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    The theoretical multistreaming limit per client is 99. That is the max value that 'Max Job per Client' in master server Host Properties can be set to.
    The default 'Limit jobs per policy' (policy attribute) is set to unlimited. 

    Factors that determine how many jobs are activated depend on Storage Unit settings - MPX and number of drives as well as MPX settings in Policy Schedules.

    It is never a good idea to go with max settings since clients are hardly ever able to generate more than 4 data streams simultaneously without suffering performance issues.
    Likewise MPX of 32 is never a good idea since media server's ability to write that many streams per device is questionable and greatly affects restore speeds.
    We have found MPX of 4 gives good backup and restore performance.

    Before you chance ANYTHING, please go through these topics in NBU Admin Guide I (in addition to above-mentioned TN): 

    Limit jobs per policy (policy attribute)

    Allow multiple data streams (policy attribute)

    Media multiplexing (schedule attribute)
    About multiplexing

    Example of using multiplexing with schedules

    Once you have read through all of these topics, introduce changes SLOWLY....
    Try Max Jobs per Client, STU MPX and Schedule MPX of 2. 
    Ensure bptm logs exist on media server(s). Review results in bptm log (performance Tuning Guide will tell you what to look fir in bptm logs).
    If performance is good, increase in increments of 2 (4, 6, 8), each time verifying and documenting results.
    At some point clients and/or media server(s) will be overloaded and performance will deteriorate.

    You need to find values that is good for YOUR unique environment.

  • belse's avatar
    belse
    Level 5
    12 years ago

    Thanks Marianne.

    I have one question do we need to enable multistreming in 2 options as per the attachement one is master server properties or only multistream in policy would suffice?

    Thanks,

    Belse

  • Vickie's avatar
    Vickie
    Level 6
    12 years ago
    Hi Belse,
     
    1) Allow multiple data streams (policy attribute) : It will allow NBU to run multiple stream/job for a policy. It must be enable to take backup in multi-stream.
     
    2) Limit jobs per policy (policy attribute) : This will set the stream/job count for a policy. Mention the count as per your requirement.
     
    3) Maximum Job Per Client (Master Server Host Properties Attribute) : This option will set stream/job count for client. It's global attribute so will be applicable for all clients. Mention the count as per your requirement.
     
    Now, among option "Limit jobs per policy (policy attribute)" & "Maximum Job Per Client (Master Server Host Properties Attribute)". Which one is having lowest count set, will be applicable.
     
    For eg. if "Limit jobs per policy (policy attribute)" is set to 5 for a policy "TEST".
     
    & "Maximum Job Per Client (Master Server Host Properties Attribute)" is set to 10 gloably.
     
    Now, backup For Policy "TEST" only 5 Stream/Jobs will run, as it has lowest job count.
     
    Thanks,
    Vickie
  • belse's avatar
    belse
    Level 5
    12 years ago

    Hi All,

    I was reading the Multistreaming configuration part and came accross with the below Unset and Unset_All Directives in Admin Guide which I didnot understand.

    Could somebody pls make me understand in simple words what is this Unset and Unset_All, its work with an example? Admin Guide is sometimes too difficult to grasp the concept :(

     

    UNSET, UNSET_ALL directives, and multiple data streams
    All policy-specific directives that are passed to a client in a stream are passed in all subsequent streams. The UNSET and UNSET_ALL directives change this behavior. These directives are recognized only if the Allow multiple data streams option is set for the policy. For more information, see “Directives for specific policy types” on page 184.
    UNSET
    The UNSET directive interrupts a policy-specific directive so it is not passed with any additional streams. The directive that was unset can be defined again later in the backup selections list to be included in the current and the later streams.
    UNSET_ALL
    UNSET_ALL has the same effect as UNSET but unsets all policy-specific directives in the backup selections list that have been defined up to this point.
    UNSET example
    In the following backup selections list, the set command is a client-specific directive that is passed to the first and all subsequent streams.
    NEW_STREAM
    set destpath=/etc/home
    /tmp
    /use
    NEW_STREAM
    /export
    NEW_STREAM
    /var
    For the set command to be passed to the first two streams only, use UNSETor UNSET_ALL at the beginning of the third stream. At this location, it prevents SET from being passed to the last stream.
    190 Policies
    Rules to indicate paths in the backup selections list
    NEW_STREAM
    set destpath=/etc/home
    /tmp
    /use
    NEW_STREAM
    /export
    NEW_STREAM
    UNSET_ALL [or UNSET set destpath=/etc/home]
    /var
    Excluding files
  • Andy_Welburn's avatar
    Andy_Welburn
    Level 6
    12 years ago

    The SET directive is a way of setting environment variables as directives in a backup selection list.

    Personally we've never used them & my only ever exposure to them was for NDMP backups.

    I doubt very much that you need to know about them ..... but I could be wrong.

     

    The following T/N (essentially part of the NDMP Admin Guide) may help (or may confuse more) but UNSET/UNSETALL could easily be incorporated into the example it gives to UNSET specific or UNSETALL the variables that had been previously SET.

    http://www.symantec.com/business/support/index?page=content&id=HOWTO53065

     

    e.g. how I would envisage UNSET/UNSETALL in the example given by the T/N:

    /vol/vol1
SET HIST = N
/vol/vol2
UNSETALL (or UNSET set hist = n)
SET DEF = 20
SET SAMPLE = all
/vol/vol3
  • Harpreet_Singh1's avatar
    Harpreet_Singh1
    Level 3
    12 years ago

    Hi Marianne,

    I remember some time back u have posted this diagram on the same topic

    Regards

    Harpreet Singh

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Thanks for reminding me! 

    The sections in the manual that I have referred to above also contain excellent diagrams.