Forum Discussion

Zahid_Haseeb's avatar
Zahid_Haseeb
Moderator
14 years ago

Netbackup Compression is not good

Environment

Veritas Netbackup Server = 7.1

Netbackup Client = 7.0

Client OS = Redhat 5.3 x86_64

Policy Type = standard

 

I am able to take backup and restore of flat files succesfully means the Netbackup Server is properly operational. My question is if I take the compress backup of a folder which aproximate size is 40GB, it size makes around 36GB via Netbackup. But when I compress same 40GB folder with the linux utility it makes a file size around 20GB or might be less than 20GB. My question is why the Netbackup is not able to compress as efficient as the OS. 

 

(      Note:

I observe the same thing in windows platform as well. Means I made a compress file of around 40GB data to 12GB from a 7zip software.     )

 

I have to compress the folder via OS utility and then backup it via Netbackup which actually increases the time of Backup. Any comment will be appriciated.

6 Replies

  • or just allowing your hardware to do it? (i.e. do you have compression ticked in the policy)

    You must bear in mind, however, that each "type" of data will compress at a different rate & also different programs are able to compress the same data at a different rate (as you appear to have found thru' your testing).

    Also note, that if you are trying to compress data that is already compressed (e.g. getting NetBackup to compress already compressed data & then hardware compression during the save) then you could encounter slow backups & the size of the saved data could actually increase.

    ***EDIT***

    See the context sensitive "Help" within policy attributes. There are 3 sections:

    Advantages of the Compression attribute Compression reduces the size of a backup by reducing the size of files in the backup. In turn, the smaller backup size decreases the amount of media that is required for storage. Compression also decreases the amount of data that travels over the network as well as the network load.

    Disadvantages of the Compression attribute Compression increases the overhead computing on the client and increases backup time due to the time required to compress the files. The lower transfer rate that is associated with compression on the client reduces the ability of some tape devices (notably 8mm) to stream data. The effects of the lower transfer rate causes additional wear on those devices.
    The savings in media and network resources, however, still make compression desirable unless total backup time or client computing resources become a problem. If total backup time is a problem, consider multiplexing. The NetBackup multiplexing feature backs up clients in parallel, reducing the total time to back them up.
    If compressed data is written to a storage unit that has single-instance store (SIS) capabilities, the storage unit may not be able to use data deduplication on the compressed or the encrypted data. In data deduplication, only one instance of the file is stored. Subsequent instances of the file reference the single file.

    How much compression can be expected? The degree to which a file can be compressed depends on the data type. A backup usually involves more than one type of data. Examples include stripped and unstripped binaries, ASCII, and the non-unique strings that repeat. Some data types are more favorable to compression.

    (The last part goes on to specify types of data that compress well & those that don't & the effects of compression on them. Plus it lists file types that are not compressed on a UNIX client)

  •  

     

     

    or just allowing your hardware to do it? (i.e. do you have compression ticked in the policy)

    I tried to take the backup of uncompress folder with the compression check box in the Attributes tab in the Netbackup Policy. which gave a me a rotio of a 40GB data to around 36GB.
     
    ( I am not compressing a folder which is already compressed ) 
     
    But i got a good ratio of a uncompress folder when I compress this uncompress folder via OS utility which may be 7zip(Windows software) or bz2 (unix).
     
    ( The compression ration is less than half of the actuall size of uncompress folder size )
  • Does your hardware also compress??? (i.e. tape drive(s))

    Try switching off the policy compression attribute & see what results you get.

     

    I mentioned compressing something that's already compressed to give you an indication of what could happen if you did

    "I have to compress the folder via OS utility and then backup it via Netbackup which actually increases the time of Backup." - policy compression still active?

    I also said that diffferent compression utilities will compress data at a different rate

  • I thought I am using the Tape Drive with default configuration and I never tried to do a hardware compression. Let me also try to check the setting of hardware compression.

    ========================================

    "I have to compress the folder via OS utility and then backup it via Netbackup which actually increases the time of Backup." - policy compression still active?

     

    Sorry here I meant that I first tried to compress an uncompress folder via OS utility and I got a specific result than I did another activity in which I tried to compress an uncompress folder via the compress check box ticked in the Netbackup policy.

     

    =====================================

     

     

    I also said that diffferent compression utilities will compress data at a different rate.

    I understand but the point is OR the main question is that Netbackup give a very bad compression rate.

     
  • 1) Quick test - backup a file/folder of known size with policy compression switched off & check results.

    2) Ok.

    3) But then compression utilities are very poor in their backup performance! ;)

  • It is recommended to use the hardware compression on the drives, eg, write to the xxcbn device (c - compression).

    I have no idea of the compression ability of NetBbackup, in almost 4 years of supporting NBU, I have never seen anybody using it, and in 7 years of 'other' backup products I have not seen it used either - why, beacuse it puts an overhaed on the servers and slows things down.

    I suspect that the reason it is not as efficient as other software compression is time and overhead - the more the compression, the longer it will take, so I suspect there is a balance between compressing really well, and time/ overhead taken to do this.  bz2  I think, comresses the best of any software compression - sure, you could have code 'similar' to this in NBU, but then people who do use software compression  would complain that the  backups are slow with compression enabled.

    My advice is either :

    1.  Log a support call and ask why the compression is less than that obtainable to the os utilities.  I suspect you will get a similar answer to mine, but as always, I am happy to be proved wrong ...

    2.  Switch on hardware compression, forget about it and enjoy life ...  

    :o)

    Martin