Restoring many small files takes to long

[Roberto_Hern_nd]

¡Hi!, I did a bere metal restore on a Solaris 10 client, everithing work fine. One thing I noticed on the restore was the restore time on the directory /var/spool/clientmqueue, this directory is the queue for send mail. I had 451485 files each file size is 1 KB. The restore of this directory takes more than 4 hours. I know this behaviour is normal for a large number of small files. I haven't found any article explaining this behavior. Does any one have an article reference explaining this mater? Regards.

[Dennis_Strom]

There is this old thread that has some good references in it.

https://www-secure.symantec.com/connect/forums/backup-many-many-small-files

[Christoph_Linde]

Okay, what exaxctly is the problem?

restoring 451485 files of 1 kb in size will take long. As you said, you know this behaviour is normal ... so aeh. What exactly is wrong?

A very short explanation, why it takes long:

The filesystem has to create all the directory entries, all the inode and space allocation, and the write of the file itself. So This takes time for a file. I do not know, what filesystem you use, or how fast your disks are ... but if you can write 100 files per sekond, it will take 4514,85 seconds, so about 1 hour and 20 minutes. If you can only write 10 files per sekonds ... it will take about 12 hours. 

So It is simply the work of the filesytem itself, that slows the process down. There is no way on per file base to come around that problem.

It is one of the most pressing problems with file backups at the moment. Terabyte of Storage in Files have to be backuped and if you need a restore, you are dead for a loooooong time.

But another question:

If this is not a very heavy loaded mail server, then your sendmail configuration is broken. Why should a not heavily loaded mailserver have half a million mails waiting for delivery? SPAM attack ? Or is your Server used as a SPAM relay?

[Roberto_Hern_nd]

Thank you for the info. That is what i was looking for.

[Dennis_Strom]

I agree that there appears to be a problem with sendmail.  with sol10 it is enabled by default.  Usually you can disable sendmail.  with sol10 that is svcadm disable sendmail.  I do this by default on all my sol10 client deployments.

[Roberto_Hern_nd]

Thers nothing wrong, infact the restore was o.k. I just want to understand why the restore of many small files takes to long. I know this is a normal behaviour, but i didn't have any reference of this on any manual or symantec support document.

As you said i have to check my send mail configuration and find the root cause of having so many files on the sendmail queue.


Thanks for the explaination.

[Christoph_Linde]

Okay, 

so is my explanation, of why it takes a long time what you expected? Or is there anything I can do for more clarification?