Large backup with millions of files and hundreds of thousands of subdirectories

[JeffFox]

We have a backup of 3 LUNS on a SANS where one LUN has 94GB with over 2 million files in over 326,000 directories.  We have a second LUN that has 115GB with 1.7 million files in over 267,000 directories and a third LUN with 138GB in over 966,000 files in over 16,000 directories.  This is the only backup that we have a problem with but also the only backup with this many files and directories.  The third LUN does complete but the other two don't and the machine crawls to almost a stop.  The machine has 4GB of memory in it but I notice initially that about 2GB of physical memory is free but the system is paging at over 2,000 pages a second. (This is not a typeo.)  Over time the available physical memory will slowly go down but the paging stays around 2,000 to 3,000 pages a second.  Since the third LUN is using more space, I assume that Netbackup is having a problem with either the number of files or the number of directories.  We originally only had 2GB but increased it to 4GB but it did not seem to help.  If we let the job continue to run; we can not access the console with terminal services and if we try to login locally the console hangs trying to login. I don't think the machine is actually hanging but that it runs out of memory and the system is paging itself to death.  We have no problem if we use a different backup product call Data Protector so I don't think we have a problem wit the system.  Is there something we are missing because of the number of directories or files?

[Stumpr2]

you are using the wrong solution for backing up the servers.

DOCUMENTATION: How to configure backup for maximum throughput when a volume or folder contains millions of small files.

http://support.veritas.com/docs/275912

[JeffFox]

Looks like you need a license to use it.  The FlashBackup is not an option on my drop down box.  :(

[Stumpr2]

oh yeah. and that is more correctly spelled licen$e :)

Seriously, you are putting backups for those servers in jeopardy as well as all the other backups in you environment that are waiting for their turn to use the tape drives if you keep trying to back them up with the normal NetBackup policy.

[Puffy]

Hi I'm exploring the use of Flashbackup.

One question is that from the document above, it says use Flashbackup.

However in the document as attached below, it says use MS-Windows-NT. In addition is this format /\\.\E: use for any raw partition backup ?

 -------------------------- 

3. Configure the policy:

3a. Policy type should be MS-Windows-NT
3b. The backup selections list should include only the volumes to be backed up via the raw partition in this format:

/\\.\E:

 --------------------------

Thanks. 

[JeffFox]

You use that if you don't have Flashbackup but then you can not restore individual files.

[CHONG_WU]

I am noticing this . For unix client like sun solaris, has flashbackup function for millions of files backup?