Forum Discussion

yobole's avatar
yobole
Level 6
11 years ago

NetBackup client-side deduplication and Multiple policies

NetBackup client-side deduplication and multiple policies 

 

I found this article 

http://www.symantec.com/business/support/index?page=content&id=HOWTO13907

And especially 

A NetBackup client using change journal support must belong to only one policy. To use one policy avoids the confusion that multiple backup settings causes. Multiple backup settings can cause conflicted update sequence number (USN) information in the permanent record “ 

 We wanted to setup two policies to backup a Netbackup client with multiple volume. Put all the smaller volumes in one policies and the larger one in one policies as the larger one taking forever to  backup … is this possible ??

7.5
Backing Up
Backup and Recovery
NetBackup
Windows Server (2003-2008)
  • Marianne's avatar
    Marianne
    11 years ago

    The maximum max jobs per client is 99. 

    This is a theoretical limit. I have never seen any client being able to handle this amount of streams.

    I would personally start with 4, then try 6, then 8. Carefully monitor cpu and memory usage on the client as well as backup throughput after each increase.  Depending on resources, you will get to a point where jobs will be queued or where performance deteriorates. Stick to the value where you got best performance.

    Remember that Max jobs per Client is a global setting.
    You can customize per client in Host Properties -> Master -> Client Attributes.
    Add/select client name to customize.

     

13 Replies

Sort By
  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    11 years ago

    Based on that tech note i guess the way to proceed it to not use the change journal (set via Host Properties of the client where you would have it set to use the archive bit)

    That may be different if you are using Accelerator though as it will use the chnage journal if it is available

    All of these however are dependant on whether the client itself actually has the change journal actually enabled

    In view of the advice and what you want it would probably be best for you to have your multiple policies, incremental backups set to use the archive bit, use accelerator too but have the change jounal not enabled on the client itself

  • yobole's avatar
    yobole
    Level 6
    11 years ago

    we do have client side dedupe , Change journal and accelerator all configured

  • Marianne's avatar
    Marianne
    Level 6
    11 years ago

    There is no need to separate volumes into different policies.

    You can add them in one policy and break them up into different streams.

    Enable 'Allow Multiple data streams' in policy attribute and divide streams as per this example:

    NEW_STREAM
    small-volume1
    small-volume2
    small-volume3
    ...etc
    NEW_STREAM
    big-volume1/folder1
    NEW_STREAM
    big-volume1/folder2
    NEW_STREAM
    big-volume1/folder3
    ...etc

  • yobole's avatar
    yobole
    Level 6
    11 years ago

    This looks like what I need I have  Enabled  "'Allow Multiple data streams' in policy attribute " but not sure what you man below . We just only Enabled "'Allow Multiple data streams' in policy attribute "

     

    NEW_STREAM
    small-volume1
    small-volume2
    small-volume3
    ...etc
    NEW_STREAM
    big-volume1/folder1
    NEW_STREAM
    big-volume1/folder2
    NEW_STREAM
    big-volume1/folder3

  • Marianne's avatar
    Marianne
    Level 6
    11 years ago

    I thought the example would be self-explanatory....

    The example groups a number of small volumes into a single stream.

    With NEW_STREAM directive you can add a big volume into a single stream, or you can break it down into smaller 'chunks' or streams. 
    You will need a good knowledge of the filesystem layout (directory structure) in order to perform this breakdown.

  • Marianne's avatar
    Marianne
    Level 6
    11 years ago

    Correct. Multiple streams is not recommended where volumes are on a single disk/device (e.g. 1 physical hard drive that is partitioned.)

    Large volumes on a SAN is NEVER on a single disk.

    Raid configurations consist of multiple physical disks that are suited for multiple read streams.

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    11 years ago

    I do have customers with that sort of data and when lots of very small files are involved a lot of streams do help even when they are on the same volume

    As you are using accelerator the track log takes care of things so it should not really be a great issue

    However, If you do change your policy you will need to do a forced re-scan (real full backup) first to recreate the track log information for the new paths

    You may also need to increase the WorkerThreads value in the /disk/etc/puredisk/contentrouter.cfg file to get the de-dupe to cope (default is 64 so it will need to go up to 128 or 256 - it should be 128 anyway when using accelerator)

     

  • yobole's avatar
    yobole
    Level 6
    11 years ago

    Thank you for all your help . yes the large 6TB is on the SAN as an RDM