Forum Discussion

Seth_E's avatar
Seth_E
Level 6
11 years ago

Millions of Files - Slow Performance

Hi there,

 

We are currently running Netbacku 7.6.0.1 with one media / master server (30 clients only for now)

File servers with 1+ million files take forever to backup with a throughput of 5 - 9 MB / sec. The backups are going directly to basic disk and even running just one job still shows slow throughput. I have a client that has been backing up since yesterday and it's only gone through 3 million files.  The file server has 6 million files on it. Other backups (such as oracle, and smaller servers( appear fine and can acheieve a throughput of 20 MB / sec +.

I can log into the client and copy a 500 MB file directly to the disk staging unit of netbackup (E:\) and the throughput is 25 MB/sec. My question is, why is throughout so slow when backing up large file servers? Is there any way to increase this? The file server has an OS drive c:\ and a data drive d:\. Multi streaming in this case doesn't seem to improve performance so, it's turned off.

my settins for the two config files are as follows:

NUMBERS_DATA_BUFFERS_DISK = 64

SIZE_DATA_BUFFERS_DISK = 1048576

I'm considering using Netbackup Snapshot Client, but, need to research it and determine if a license is required.

 

Any advice or assistance is appreciated.

 

EDIT: I forgot to mention that differencial backups are fine and don't take long since I'm using Journaling. This issue pertains to full weekly backups.

 

  • This is a know dilemma for systems with many small files. Time used for opening,reading and closing many files simply take a long time when repeated enough times.

    The best weapon is Netbackup deduplication (MSDP) and Accelerator. 

    The license for 7.6 should include using MSDP as far as I can recall - but I may be wrong.

    Abduls Blog about accelerator:

    https://www-secure.symantec.com/connect/blogs/frequently-asked-questions-netbackup-accelerator

    https://www-secure.symantec.com/connect/blogs/frequently-asked-questions-netbackup-accelerator-part-ii

  • This is a know dilemma for systems with many small files. Time used for opening,reading and closing many files simply take a long time when repeated enough times.

    The best weapon is Netbackup deduplication (MSDP) and Accelerator. 

    The license for 7.6 should include using MSDP as far as I can recall - but I may be wrong.

    Abduls Blog about accelerator:

    https://www-secure.symantec.com/connect/blogs/frequently-asked-questions-netbackup-accelerator

    https://www-secure.symantec.com/connect/blogs/frequently-asked-questions-netbackup-accelerator-part-ii

  • The slow performance is likely just due to the number of files.  Flash Backup would be a better choice when you have that many files and can't get a fast enough throughput to complete within your backup window.  You will need an Enterprise Client license for Flash Backup.

    Other options would be to use NetBackup Accelerator and Media Server Deduplication, which should improve the throughput on full backups, but requires Data Optimization license; or synthetic fulls, where you run one complete regular full, then run daily incrementals, and a synthetic full every week after that.  Synthetic backups might require a different license, but I'm not sure.

  • When faced with a similar problem, we used a low-tech solution of chopping the single drive letter (D:\) into 4 different policies, multi-streamed, with backup selections and exclusions to make each policy unique. That, combined with 10Gbit ethernet upgrade on Media Server and Client (we already had MSDP), was able to reduce the Full backup elapsed time from "intolerable" to "tolerable". The result is a weekly Full that previously ran into Monday afternoon, now finished early Sunday AM.

    While researching the issue, I also considered disabling 8-dot-three naming on the NTFS filesystem to improve file open time, but disabling 8-dot-three would required a conversion process that management was not willing to undertake.

     

  • I was thinking of this solution as well, but, I didn't want to run into the problem where new folders are created on the root of d:\ and as a result, will not be picked up by the split. I'm not too happy with synthetics in terms of performance but, will give this a shot.

     

    I'm thinking to do:

    cumulative incrs Sun - Fri

    Synthetic full - Sat morning

    Full - Monthly on last day.