Forum Discussion

Glen_Harrison's avatar
Glen_Harrison
Level 4
13 years ago

Catalog compression

Hi

We are running nbu 7.0.1 on an HP-UX 11.31 IA64 master server our catalog is getting to be about 800GB and it is taking about 14 hours to backup.

 

Does catalog compression work well and is it just a simple to enable as checking the Compress Catalog entry in the master server properties and specifying a number of days? What do most people start out with for a number of days? I was thinking maybe 30 or 60 to start.

7.1.x and Earlier
Backup and Recovery
NetBackup
  • teiva-boy's avatar
    teiva-boy
    13 years ago

    If you speak with some of the NBU architects within Symantec like Dave High, they'll strongly recommend not getting over 1TB in size.  That at 800GB, you should be standing up a new NBU domain, and starting new, or splitting the catalogs.

    Backup, performance, and recovery (catalog import) gets very restrictive as you get to this size.  

  • Glen_Harrison's avatar
    Glen_Harrison
    Level 4
    13 years ago

    Also once enabled is there a separate compression process or does it happen as part on image cleanup?

  • mph999's avatar
    mph999
    Level 6
    13 years ago

    EEkkk ... ok from memory

    Should work fine ..

    Happens automatically , I think after a backup

    Compress in stages, first how old is the catalog, for eg. let us say 3  years.

    I'd compreess just 1 month , and see how long it takes, then judge it from there ..,

     If you  compress too much it one go  you may overload the system

    All covered in the admin guide.

    Martin

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    13 years ago

    In general and in my experience I have always advised against catalog compression.

    It can have an impact on performance as it is quite a complex process and takes even longer if you decide to uncompress it again later.

    This tech note and all of those linked within it explain it all - but I would still advise against it - after all, disk is pretty cheap these days!

    http://www.symantec.com/docs/HOWTO33397

  • mph999's avatar
    mph999
    Level 6
    13 years ago

    Mark makes some good points - another option is Catalog archiving.

    You set up a special policy (all explained in the Admin Guide) and this will backup the catalog xxx.f files (the big bit of the catalog) to tape, then delete them from disk.

    Providing (and I'm serious) you make 2 or 3 copies of these tapes it is a good way to reduce the catalog size.

    As a matter of interest, how big is the catalog ?

     

    Martin

  • Glen_Harrison's avatar
    Glen_Harrison
    Level 4
    13 years ago

    Thanks for the responses

    Our catalog is 816GB as of this morning, the backup time of the catalog is what I am really trying to shorten. It currently takes about 14 hours to backup our catalog over the network to a remote NFS share. If I backup to local disk it takes about 12 hours.

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    13 years ago

    Dont forget you can do Incremental catalog backups so may only need to do a full once a week

    catarc can help but it depends on what your plans for the near future are as you dont want to go removing chunks of the catalog if you have plans to move things.

    It is also always worth reviewing your retention periods and if they are valid - the longer you keep stuff the larger your catalog grows.

    I am still against catalog compression though!

    Hope this helps

  • teiva-boy's avatar
    teiva-boy
    Level 6
    13 years ago

    If you speak with some of the NBU architects within Symantec like Dave High, they'll strongly recommend not getting over 1TB in size.  That at 800GB, you should be standing up a new NBU domain, and starting new, or splitting the catalogs.

    Backup, performance, and recovery (catalog import) gets very restrictive as you get to this size.  

  • Sammeer_Pednega's avatar
    Sammeer_Pednega
    Level 4
    13 years ago

    Hi Mark

    Would suggest the following

    1) Reduce the frequency of image cleanup

    2) Compress / move / backup of Catalog folders for each server that has been decommission or migrated

    3) Delete all imported images