Anybody use compression with VTL's?

[todis]

I'm currently using an Overland REO 9000 with three VTL's. When I initially set it up I chose not to use compression because I thought it would be too slow. The REO's OS handles the compression and I've always heard never to use software compression only hardware. I'm running out of space now so I'm thinking about enabling compression for one VTL and trying it out. Is anybody using compression enabled from the software/OS that created the VTL's and if so what's your performance like? Thanks

[todis]

vtl

[TomerG]

I'd imagine that compression on the VTL should be moderately ok. When they say not to enable it in the 'software', they usually mean on the backup client software on the client, so that it won't affect your application. in this case the VTL, which effectively is an appliance, should theoretically be able to handle it, however 

That said, there seems to be a consensus online that some VTLs do get worse performance when enabling this. Here is asnippet from a vendor who is trying to sell you on the fact that their VTL's compression is done in the hardware: "Unlike many VTL solutions that offer software-based compression, which may actually slow performance to slower than that of a physical tape drive and effectively destroy the benefits of VTL, VSE offers hardware-assisted compression, which maintains the high speed of VSE. " (see http://www.compactpci-systems.com/news/db/?1566; no I'm not a salesperson! :)

You'd really want to do testing to guage the performance difference (simple enough to do a quick test of a large file backup with and without from the same client around the same time of the day). I'd be interested to hear of the results.

[todis]

Ok -- Thanks for clearing up the VTL compression. I assumed since the VTL was virtual and software based that compression would be also. So at this point I should try enabling compression on the VTL but not in Netbackup correct? Doing so would compress the data twice, once by Netbackup and once via the VTL? I imagine doing both would slow things down.

[Stumpr2]

We define the tapes when we add them to emulate exactly what will be used for restoring.

[Rakesh_Khandelw]

Compression at NetBackup level is software compression, compression at VTL
level is hardware compression. Typically, VTL's have compression cards but you need to check the compression switch in management GUI to start utilizing compression and VTLs are powerful enough to handle this compression.

I suggest you turn it on and you may see a better performance.

[todis]

Tomer, I will look into setting up a test and will post the results. I feel better about enabling compression on the VTL now and hope it isn't a drastic performance difference...

[todis]

Well, now my backups are taking about twice as long to backup. Jobs that normally took about 18hrs are finishing now in 1day, 15hrs. I haven't yet done a detailed comparison but my throughput has dropped significantly. I only have compression enabled on the VTL. I know a previous post said to enable compression in both NBU and the VTL. Wouldn't this decrease my performance even more? I'm scratching my head at this point....

[CY]

You posted originally:

"I'm currently using an Overland REO 9000 with three VTL's. When I initially set it up I chose not to use compression because I thought it would be too slow. The REO's OS handles the compression and I've always heard never to use software compression only hardware. I'm running out of space now so I'm thinking about enabling compression for one VTL and trying it out. Is anybody using compression enabled from the software/OS that created the VTL's and if so what's your performance like? Thanks"

What you've always heard is correct - don't (never) use software compression.  The only compression that has little impact to backup performance comes from hardware compression, which is usually an ASIC chip on the library to handle the compression calculation - both the ADIC physical tape library and the EMC EDL virtual tape library that we use have the hardware compression chip.

And I would never enable compression in both NBU and the tape library (VTL or physical).  They pretty much use the same compression algorithm, so once a file is compressed, it is very difficult to be compressed again and sometime you even get negative results.

[todis]

CY -- thanks for you response and clarification. At least now I know from experience not to use compression managed by an OS...
That leads me to another question: If I disable compression on the VTL's will the images that were created using compression still be valid?

[Amit_Karia]

How long are you going to keep data on VTL.. are you going to duplicate images from  vtl to tape library or, going to retain it as long as image does not expire.. ?

[todis]

Everything gets duplicated to tape but there a several images that we keep on disk for quick restore that don't expire until two weeks later.

[TomerG]

 Sounds like you guys have the general right ideas. Compression on both sounds like a generally bad idea.

In the olden days compression feature was there because of limitations in network bandwidth. If you were doing remote backups over a WAN or slow connection, it could be that compression would make a negligeable impact on system performance (assuming machine is fast enough), and could make a small gain in network use (since we'd be sending less data overall). That said, nowadays network bandwidth is not usually the bottleneck, or even so the small gain in compression over the CPU overhead is usually not worth it in the long run.

Also, backups over WANs is answered better by other solutions, like NetBackup Puredisk, or even things like Backup Exec's Continuous Data Protection (CPS).

[todis]

Tomer -- thanks for your response.

Does anybody know if removing compression on the REO VTL's will erase currently compressed images?

[scorpy_582]

Hi,

I am not sure on this one, but I dont see a reason why removing compression would hamper the current images.

[todis]

Well, after all the images are duplicated to tape I'll remove the compression and see what happens. Wont be able to try it until probably Wed.

[TomerG]

The previous backups' information is stored in their catalog information (on the master) and backup header (on tape), so each individual job remembers whether or not it was compressed or not. If you change the policy it has no effect on the recoverability of previous backup images.

[todis]

Tomer --thanks-- I want to make sure we're on the same page. I do not have compression enabled within Netbackup. Compression is enabled in the VTL OS that runs on my Overland REO 9000. Does your reply still hold true in this case? It sounds like you're referring to compression enabled in NBU.

[TomerG]

 yaa, you should probably test... not sure what happens with the VTL compression suddenly turned off

[CY]

Hi todis,

I haven't checked this thread for many days, so sorry I did not follow up with your question.  

I don't think the VTL will erase the existing (compressed) backup data when you turn off the compression feature.  However, I'd suggest you to check with the VTL's vendor.  Since it is S/W based compression, the compression binary/code does exist whether you enable it or not.  I think they could have the feature "enabled on demand" so the VTL still can call the compression/decompression algorithm when it's necessary to recover old backup images using compression.

Hope this helps.