Backup Solution

[MikeM11]

Hi,

I have been banging my head against this issue for a while and wondered if anyone could see a solution that I have not yet considered:

I have a Client that is Firewalled off where traffic cannot pass through it to a NetBackup Media server so we have installed NBU Media server (7.1.0.3) on this Client and provided access via fibre to the tape drives.  On a Full backup we capture over 17.5 million files which are .WAV files but due to the directory structure of one file per folder (i know), we are seeing speeds of around 9MB/sec.

This one stream currently takes almost 24 hours to complete a full and the archive is increasing by around 100GB every month.

Currently we run a standard Weekly Full and Daily Differential Incremental.

The backup goes direct to tape as .WAV files do not dedup and we have no advanced disk available for staging.

We are unable to use FlashBackup as the data cannot be snapshot due to the vendors requirements.

Any assistance would be helpful.

Many thanks, Mike

[Yogesh9881]

Let us know more about " data cannot be snapshot due to the vendors requirements."

What storage you are using ??

[MikeM11]

The vendor advises that they "do not allow" snapshotting of files as it may interfere with their application.  The business decision was therefore to not allow open file backups and disable VSS.

The files are stored on a SAN LUN presented to the Client.  The NetBackup policy then uses a Media Manager storage unit to commit the backups to tape.

[mph999]

What about a raw backup of the 'disk'.  Will only be able to restore the whole disk, not individual files, but should be quicker.

M

[V4]

If it's SAN LUN. try with NDMP option.

[Yogesh9881]

???

how ?? 

[V4]

yogesh don't give a sigh you are not aware of NDMP backup  ;)

One of customer had similar issue with millions of files . just enable NDMP at filer level and add NDMP license keys to NBU. refer NDMP guide for how to. and you are through.

[RonCaplinger]

1. Switch to disk storage units instead of tape.  At 9MB/sec, you are not streaming data and the backup is likely having to wait for the tape drive to constantly reposition the tape.  Back up the millions of small files to disk first, then duplicate the one big backup image to tape (Storage Lifecycle Policies are perfect for this).
2. Break down the full backup by directory, if possible, and use multiple backup policies with different schedules for the fulls.  For instance, if the directory structure is by date (or alphabetical):

           F:\AppName\DailyFiles\2010-01-01\00_00_00

           F:\AppName\DailyFiles\2011-01-02\00_01_39

           Then create a policy that backs up, say "F:\AppName\DailyFiles\2010*", with the full backup on Monday.  Create a second policy that backs up "F:\AppName\DailyFiles\2011*", with a full backup on Tuesday, etc.  You can even create multiple streams within the policies' backup selection lists:

            NEW_STREAM

            F:\AppName\DailyFiles\2010-01*

            F:\AppName\DailyFiles\2010-02*

            F:\AppName\DailyFiles\2010-03*

            NEW_STREAM

            F:\AppName\DailyFiles\2010-04*

           etc.

           This is what we had to do with a number of our backups that contained millions of files.

      3.  Push your users and their vendor to add an enhancement that will archive older data off the current production disk storage into .zip or .cab files, while retaining access to that old data.  Symantec is now doing this with Enterprise Vault.  We went from 85 million small files to about 35 million medium sized files in EV using this process and our backups are almost at an acceptable speed now.