Slow backup of large number of small files

[Bill_Clark22]

Running Veritas BUE v16 and encountering an issue when backing up servers that have a large number of small files.  For example, one of our file servers has 4.3 million files (mostly Office docs, jpg, etc.) in 600K folders for a total size of 1.1TB.  This takes 18 hours to backup to disk at a rate of 1127MB/min.  We do this job once a week, then run nightly differential backups against it through the rest of the week.  For this disk-to-disk job, we aren't doing compression or encryption. This is not an abnormal issue with this one job.  We have another job of similar size and file composition that takes 14 hours to complete.  Jobs of similar overall size, that are setup to backup entire VM's take 1/2 the time to complete.

How can I get more speed out of the data backup jobs?  Or is this simply how it is due to the nature of the files being backed up?  Thanks. 

[Larry_Fine]

Huge quantities of small files have always been low on the performance scale due to all the directory access required.  Large files have better throughput due to less directory overhead.

[Bill_Clark22]

That's what I was fearing the issue was, but hoping that maybe there was a setting I'm missing in the job configuration.  On these two servers with the large number of files, we do a full VM backup on Saturday, but then we have to do a data-only backup on Monday so we can have that base-line backup for the differentials.  Guess my best bet is to tweak the schedule a bit and do the full data-only backup early Sunday morning so it doesn't overlap into the work week.

[pkh]

Using the modified time method will be faster than the archive bit method

[Otso]

I'm running  Veritas Backup Exec v20 and I'm having issues when backing up File server that have a large number of small files (excel , word ,pp ,pdf ,pictures and video files). Its taking 3 days to backup the 3 TB server at a rate of 616.00MB/min -I have tried to defragment/check disk and reconfigure the job, increase memory on the File and backup server ,check disk on the Hyper-V but no joy .When I do a copy 3GB file from the backup server to the file server it take 4mins or less. Both the File and Backup server are sitting on the same Hyper-V with fibre NIC ,I notice that it's only slow when backing up the small files only on D: 

The file server (running Windows 2016) is a VM as well as the backup server and the tape autoloader is LTO7 ,storage controller   Dell 12Gbps SAS HBA and it redirected to the Backup Server VM running Windows 2016 

Please help ?

[PJ_Backup]

There is no one easy answer to this problem, however I would recommend reviewing the following links. For instance the Tuning and Performance Guide has "Tips for backing up many small files" (Chapter 3)
https://www.veritas.com/support/en_US/doc/40056030-98804569-1 

These tuning videos by a BE partner also contain good information:
https://blog.pingus.de/2016/04/14/backup-exec-performance-series-part-1/ 
https://blog.pingus.de/2016/05/30/backup-exec-performance-series-part-2/ 

Additionally the BE Best Practices guide can be useful:
https://www.veritas.com/content/support/en_US/doc/72686287-140583204-0/v70445037-140583204

Lastly it may be worth experimenting with more jobs in parallel rather than just the one job serially.