Forum Discussion

LilmanHSU's avatar
LilmanHSU
Level 3
11 years ago

Large Job Spanning Multiple Drives

Hi,

 

We have an Quantum i500 Tape Libraray with 6 drives and lots of available slots.  We have a backup job that we would like to create to backup approximately 150TB on a single volume.  Volume is E:\*.*

Subfolders within the root level of the E drive range from 80TB to 10TB in size.  We are exploring NetBackup as a possible solution, but wanted to know if there is a way to setup the job to backup the E:\*.* Folder across multiple tape drives at the same time vs. just a single drive.  We're also concerned about splitting up the job. 

Job1 Backs up Folder 1

Job2 Backs up folder 2

Job3 Backs up Folder 3 & 4

If the users who use this folder create a Folder5, we won't know it is there and won't back it up.  Again, this is my assumpiton, I welcome all feedback and suggestions as we are evaluating NetBackup at the moment.

 

Example for Illsustration Purposes:

E:\Folder1 = 80TB

E:\Folder2= 30TB

E:\Folder3=10TB

E:\Folder4=30TB

7.6
Backing Up
Backup and Recovery
Best Practice
Evaluating
NetBackup
Tip-How to
Training
Troubleshooting
Windows Server (2003-2008)
  • SymTerry's avatar
    SymTerry
    11 years ago

    Hello, 

    NetBackup has a feature called Multistreaming. Multistreaming sends data from a single client to multiple tape drives. This assumes that you have a high end client that can send data faster then the tape drives can write to improve efficiency.

    Check out TECH31512: What is Multistreaming and how is it turned on? Also see TECH10085: What is the difference between multiplexing and multistreaming?

  • RamNagalla's avatar
    RamNagalla
    Moderator
    11 years ago

    hello,

    what about the number of files.. are they in small size and large number of files.. or normal files..

    i guess you need to look for the  below options

    1) flash backup

    2) client side dedup with accalator , but if you have the tape dirves it is not supported...

    3) SAN media server(make the client as media server)

    splitting into multiple jobs will help you to take different tape drives. but i wonder how many days its going to take to compleate the backups.

  • SymTerry's avatar
    SymTerry
    Level 6
    11 years ago

    Hello, 

    NetBackup has a feature called Multistreaming. Multistreaming sends data from a single client to multiple tape drives. This assumes that you have a high end client that can send data faster then the tape drives can write to improve efficiency.

    Check out TECH31512: What is Multistreaming and how is it turned on? Also see TECH10085: What is the difference between multiplexing and multistreaming?

  • RiaanBadenhorst's avatar
    RiaanBadenhorst
    Level 6
    11 years ago

    Hello,

     

    Do you seriously have a 150TB lun assigned to a windows server and its all accessed via you E-Drive?

     

    What is kept on this drive, in terms of file type/application?

  • RonCaplinger's avatar
    RonCaplinger
    Level 6
    11 years ago
    1. Create one policy with Backup Selection=ALL_LOCAL_DRIVES
    2. Create one policy with Backup Selection=E:\Folder1,
    3. Create one policy with Backup Selection=E:\Folder2, etc.
    4. In client properties, EXCLUDE each of the folders (E:\Folder1, E:\Folder2, etc.) that will be backed up individually, for "<<All Policies>>" and "<<All Schedules>>".
    5. In "Exceptions to Exclude List:", add one exception for each of the above folders, for the specific policies you created in steps 2, 3, etc.

    How this works:

    First, the ALL_LOCAL_DRIVES policy will backup everything, including the entire E: drive,

    BUT, because you entered exclusions for the known folders, they will not be backed up in the ALL_LOCAL_DRIVES policy,

    BUT, the individual folders *will* be backed up in their own policies, and you can schedule the full backups to occur on different days/weeks, so you aren't backing up the entire 150TB all at once.

    AND, if a new E:\Folder* gets created and they don't tell you about it, it will still get backed up by the ALL_LOCAL_DRIVES policy.

  • Yogesh_Jadhav1's avatar
    Yogesh_Jadhav1
    Level 5
    11 years ago

    Further to Ron's suggestions - Enable multi-streaming so that each folder backup will get a dedicated drive and Backup will take considerable less time compared to a single stream backup.

  • Marianne's avatar
    Marianne
    Level 6
    11 years ago

    You can have E:\Folder* in Backup Selection with 'Allow multiple data streams' in the policy attributes.

    This will create a new job for each folder and will therefore not matter if Folder5 is created.

    Ensure that Master Server Global Properties has 'Max Jobs per Client' to a value of greater than 1 (4 is a good average).

    If you have MPX=1 in Schedule, each folder will go to a different drive.
    If you do not see backup speeds that are close to drive capabilities, you may want to multiplex more than one stream to a single drive:
    Ensure that the Storage Unit Properties is set to MPX > 1 (4 is good average) and MPX in schedule > 1.
     

  • LilmanHSU's avatar
    LilmanHSU
    Level 3
    11 years ago

    Haha.. everyone seems to be geuninely surprised we have a volume this large on a windows server.  Allow me to elaborate and perhaps this will help sort out the maddess to our reasoning.

    This LUN is a 6gb Fiber connection to a Quantum StorNext File System which contains probably around a million or more files including video, audio, pictures and all sorts of other media.  

    The client can be whatever we build it to be.  But this multistreaming feature is intersting.. I'll have to research that more.

    thanks everyone!