Forum Discussion

CoastMan's avatar
CoastMan
Level 2
9 years ago

Folder and file structure slowing down backup to tape

My apologies if this description is a little lengthy.

The Scenario.
We run BE 2012 on a Windows Small Business Server 2011 (essentially Server 2008 R2). We have a proprietary piece of software for our business that creates archives of documents within a folder structure on one of our data drives. The folder structure is complex - the software uses an alpha-numeric directory structure to store the files using client references - and contains 1,414,887 files in 777,964 folders and totals some 180GB and is always on the increase. This structure is immutable.

Our problem, which is probably apparent to you already, is that these millions of files and folders are slow to back up - about 300 - 600 MB/min. Our tape backup now lasts around 16 hours and is run daily. I would like to reduce this time, but cannot easily change the folder structure or location.

I have wondered if the following is possible and am looking for affirmation of this or, if not, any suggestions as to an alternative. We have limited financial resources so making use of what we have is my first port of call.

My thoughts: even if data volume is the same, single large files back up very much more quickly than thousands of small files. If I remove these archive folders from the tape backup job and, instead, create a new backup to disk for these files (weekly full with daily diff) I would get three or four very large BKF files on disk. Could I then successfully back these up to the tape drive in the normal daily backup?

Thank you.

2012
Windows Server 2008 R2

2 Replies

Sort By
  • jurgen_barbieur's avatar
    jurgen_barbieur
    Level 6
    9 years ago

    First thing to do, is to find your bottleneck.

     

    when your bottleneck is known, than you could think about the best way to eliminate this bottleneck

    how to find your bottleneck? monitor your target disk, cpu,  network, specify the type of your tape drive to tackel your tape speed, ...)

  • Colin_Weaver's avatar
    Colin_Weaver
    Moderator
    9 years ago

    Typically file level backup products (such as Backup Exec) will go slower if a huge amount of small files is involved

    The solution would usually then be a block or image level backup of the volume using something like Veritas System Recovery (although you won't be able to write to tapes with this product).

    Note: If the server in question is virtualized then Backup Exec can do an image level backup