We're getting better results with Winzip! 5x faster, 3x more efficient

[Ross_Smith]

We've spent thousands of pounds and months of time configuring BackupExec yet it seems that our old backup system of Winzip and batch files may be the superior setup.

Using Winzip, 54Gb of data can be backed up in 2hrs 40mins and occupies 9Gb on our NAS device.

Using BackupExec, that same data set takes 13 hours to back up, and occupies around 30Gb on the NAS device.

So our old Winzip system gives us a 5x gain in performance, and a 3x gain in storage on the NAS box... and I bought BackupExec as an 'upgrade'.

Frankly I'm appalled to discover this. BackupExec is one of the market leaders in backup technology, Winzip is not in any way shape or form designed as a backup solution. To find out Winzip is this much faster than BackupExec is a huge shock, especially when one of the selling points of BackupExec is the compression ability of the remote agent.

I'm now at the stage that I'm wondering whether I should scrap the idea of using BackupExec and concentrate on updating our Winzip batch files instead. We require a daily full backup of this data set and there simply is not enough time for me to achieve this reliably with BackupExec.

I'm 99% certain that these performance figures can be accounted for by the remote agent achieving negligable compression and transmitting the files at full size. The slight compression we do see I believe is occuring on the server itself prior to storage of the data on the NAS box.

My guess at compression figures:
Winzip - 6:1
BackupExec Remote Agent - between 1:1 and 1.2:1
BackupExec Server - 2:1

Does anyone have any thoughts as to whether it's possible to improve on BackupExec's compression?

I'm guessing the remote agent compresses small chunks at once before transmitting over the network. Adjusting this so that it looks at a larger sample of the data may be all that is needed, but I've no idea whether this is possible.

Ross

[Amruta_Bhide]

Hello,

For the performance problem please go through the following documents:
Reasons why the data throughput rate can be slower than the theoretical maximum when backing up to tape media http://seer.support.veritas.com/docs/231488.htm

What Backup Exec settings can be modified to reduce the amount of time it takes for a backup to run?
http://seer.support.veritas.com/docs/249090.htm

How to correct slow backup performance, slow virus or pre-job scans, and agent initialization problems on fragmented Windows NT, Windows 2000, and Windows 2003 server partitions http://seer.support.veritas.com/docs/237444.htm
------------------------------------------------------------------------
For the compression issue :

Hardware compression is done by the device itself. When you use hardware compression option on Backup Exec (BE), BE commands the hardware to go ahead with the compression. However the hardware compression depends on the device. The device should support Hardware Compression. We advice you to contact your Vendor for more information of the same.

Software compression is done by BE. It is said to achieve of 2:1 but that ratio is hardly ever achieved. Compression can be highly variable depending on your input data, as the software compression depends on the type of file or data you are trying to back up. For example: Already compressed files cannot be compressed any further. And the compression for normal '.ttext files will be more than the compression of '.gif' or database files.


You can refer to the following article to understand this concept better.



http://seer.support.veritas.com/docs/199542.htm


You can also go through the following article which will help to check that the data is being properly written to the media.




http://seer.support.veritas.com/docs/246958.htm

***********************************************************************
************************************************************************
NOTE : If we do not receive your reply within two business days, this post would be marked ‘assumed answered’ and would be moved to ‘answered questions’ pool.

[Ross_Smith]

I'm afraid none of those articles are relevant. I don't have a problem with network transfer rates, and BackupExec is configured to be as efficient as possible. My problem is simply that the BackupExec and it's remote agent are achieving negligable compression.

There's no problem with throughput. Both BackupExec and Winzip are transferring the files at a good rate over the network. The problem is that BackupExec's poor compression means that it is transferring 6 times as much data. This correlates very closely with the increased amount of time taken.

At the server side, BackupExec's software compression is working at the quoted 2:1 ratio, so we're saving some space on the NAS box. The problem is I know that data consistently compresses at 6:1. By swopping to BackupExec we've had to reduce by two thirds the amount of backup jobs we can store.

I would like to know if there is any way to improve the compression within BackupExec. Both the remote agent and server are being put to shame by an old shareware utility.

Ross

[Steve_Mullenix]

I've felt this way for awhile now. Pretty bad that they managed to screw up tar eh? :)

[priya_khire]

Hello,

What are you backing up and wht is the amount of data? It seems that you are using software compression? Are you backing up on a backup to disk folder? In case you are using a tape for backups,which is the tape and what is the compression ofr the tape? We recommend you to use hardware compression in case you are backing up on a tape.

Do elaborate on the above factors.

Regards.

[Mark_Greenberg]

You could, as a test, try compressing with Winzip, then backing up just the zip files from the remote computers with Backup Exec. If all of the sudden Backup Exec can do the job in Winzip-type times, then you know the problem is that Backup Exec isn't compressing the files as much. If its still slow... the problem probably resides somewhere else (high overhead with Backup Exec somewhere?).

[Gauri_Ketkar]

Hi,

We would like to know which compression type you are using ?

Please update us on the issue.

Thank you.

NOTE : If we do not receive your reply within two business days, this post would be marked "assumed answered" and would be moved to "answered questions" pool.

[Ross_Smith]

Sorry, we've had a lot of problems in the workshop today so I've not had time to work on the backup server.

In this case we're using software compression. As I already stated, we're backing up data to a NAS box and I find hardware compression is restricted to tapes as a rule...

I am planning to test the idea of zipping up a backup to test the transmission speed later today. I'll post the resulst as soon as I have them. Personally I'm betting that BackupExec's backup times will match or beat those of Winzip once the files are already zipped.

Ross

[Ross_Smith]

Ok, I have some interesting figures.

I ran the tests on a 423Mb test CAD model which consisted of 46,346 files.

Winzip can compress this to 77Mb (about 5.5 to 1), and completes the backup in 2m10s. 2m for the compression, around 10s for the transfer.

BackupExec compresses this to 278Mb using software compression (according to the Media properties), (about 1.5 to 1), and takes 12m17s for the backup.

Curiously, if I ask BackupExec to backup the zip file, it takes 11s. Extrapolating from this, BackupExec would seem to be capable of backing up 423Mb in 60s flat (12 times faster than it's current results).

This would imply that my problems are not actually with compression, but are more likely caused by the amount of files contained within this backup. It seems this is causing a 12x decrease in the data transfer rate within BackupExec.

Incidentally, this still means Winzip is faster, just for a different reason. Winzip catalogued, compressed and stored all 46,000 files in 2m. BackupExec took 12m for the same job.

Ross

[Mark_Greenberg]

Ok, try this instead. Try zipping the files using zero compression. So the resulting zip file should be around 423MB, but all as one file instead of 46,346 files. See what the difference is then. I'm guessing it'll be something like 40s for the backup (extra seconds to account for backing up 423MB instead of 77MB), and the reason BackupExec takes so long for a normal backup is the overhead for it to find and backup each of those forty six thousand files.

[Ross_Smith]

Ok, Winzip with no compression creates the archive in 2m10s (actually 10s longer than creating it with compression).

This creates a zip file that's 438mb in size. BackupExec can back this up in 21s flat. Windows explorer takes 45s to copy this file, so BackupExec actually has faster data transfer rates.

So the problem is definately the number of files and not the total size. But that's no excuse and Winzip is still proving far faster than BackupExec. Winzip can handle all 46,000 files in 2 minutes. BackupExec takes 6 times longer.

Now for a single model that's not too bad, but bear in mind we have a lot of these. Today's count is 249 models of varying sizes, total size around 50Gb. I'm not even going to try to get an accurate count of the number of files... let's just say 4.6 million seems a reasonable estimate.

Winzip can cope with all of this comfortably, taking 2-3 hours for a full backup of the lot. Doing the math means BackupExec needs 12-18 hours to complete our overnight backup, and this is confirmed with real-world results.

Using Winzip, we can comfortably run a full backup every night, and keep a 31 day history without our backup server even breaking a sweat.

We can't even contimplate this with Backup Exec. It simply is not fast enough to complete our overnight backups in time. Differential backups are not an option since they do not track file deletions or renames and will result in corrupt models. Moving to BackupExec means we can only take backups at the weekends.

For the time being, I'm running two backup systems in parallel. Winzip is our main backup solution, and I'm using BackupExec to give me a tape copy at weekends...

I still want to move away from Winzip to a 'proper' backup solution and I'm waiting to hear if anyone at Veritas has any ideas as to how we can improve the performance of BackupExec. I'm still appalled that it's being beaten by a shareware utility that's been around longer than I have... Veritas, you should be ashamed.

Ross

[Ross_Smith]

Bounce!

No reply from Veritas? Come on, there were three of you queuing up to answer me earlier? Is there any way to improve the performance of BackupExec since it's being soundly thrashed by Winzip?

I've already had an e-mail reply from Winzip about using their product for backups, don't tell me I'm going to get better support from them as well as better performance?

Ross

[John_Chisari]

Ross

There is not really much else you can do with BE to speed up or compress the data more than what it is doing.

BE (and I will say other backup programs) have always had problems backing up alot of small files.

BE used to have the Intelligent Image Option, which was used primarily to speed up the backup of alot of small files - however, this option is gone in BE 10.

The only other suggestion I can make with regards to BE 10, is to look at the Synthetic backup option.
You mentioned that differential backups are not an option as they do not track file deletions or renames, and will result in corrupt models - Synthetic backups actually does this - allowing you do an incremental backup of data forever - then once a week (or more frequently) create a synthetic full backup of the incremental data. Using the last synthetic backup will then allow you to restore the server to the exact same point as the last incremental.

[Ross_Smith]

Ok, so in order for BackupExec to compete with a shareware program I need to spend what, £2,500 or more to add extra functionality? (I believe the synthetic backup option is licenced per target server and we've 13 targets for this particular job).

Even ignoring the cost issues, synthetic backups only give me a full backup (tracking file deletes, etc), AFTER I merge the incremental backups, so this only becomes a possibility if I commit to creating a synthetic full backup every day. Is anyone from Veritas prepared to say how quickly that is going to run? I don't want exact times, just give me ball park figures for an average backup. Is it approximately the same as regular full backups, twice as fast, what?

It looks like my only option it so spend an extra £2,500 (at least) to maybe, if I'm lucky, bring BackupExec to a point that it can compete with Winzip...

I'm really not interested in excuses - It's no good saying that BackupExec (and other programs) have always had problems backing up a lot of small files. Winzip seems to manage just fine. To be honest, what you've just said tells me that Veritas know this is a problem in BackupExec and that makes me wonder why it hasn't already been fixed? Hell, as far as I know this issue with small files isn't even documented anywhere. It certainly wasn't mentioned in any of the performance articles linked at the start of this thread...

I don't expect to pay this kind of money for a program that falls flat on it's face when faced with certain types of backup... it makes me wonder what other situations are going to cause me problems in the future.

Please, please someone at Veritas take ownership of this and sort out this performance issue. For all I care, just licence Winzip and provide an 'extended compression' add-on for BackupExec. For a decent price, I'd buy that. It can't be too hard to compress things in chunks at the client end, transmit the compressed file over the network & just interrogate the zip file on the server to build the catalog?

Ross

[Ross_Smith]

Update: I just had a thought and checked some of our old logs. It seems that we used to get around 8.4Gb/hr out of BackupExec, since upgrading to V10 we're getting 4.1Gb/hr.

Any thoughts on this? 8.4Gb/hr is still not ideal (it's still 3x slower than winzip), but at least that is fast enough to complete overnight.

Ross

[John_Chisari]

The synthetic backup option which is part of the Advanced Disk-based backup option is licensed per media server.

Synthetic backups as you say merge all the incrementals since the last Synthetic backup - how long is this going to take? Probably very similar to the amount of time it takes to duplicate a backup from disk to disk - quantifying how long a backup is going to take is nearly impossible.

Synthetic backup is used to cut down your backup window and cut down the usage of your network - which is does quite effectively.

I am not making excuses, why would I? It is the way it is.
Can Winzip backup an online 100gb Exchange database to tape in less than an hour and tell Exchange to flush the committed logs?
How does Winzip handle the backup and restore of the Active Directory?
Can Winzip backup a Novell NDS?
How does Winzip handle open MS Word files when creating an archive - will this file be usable if backed up in an open state?
When is Winzip going to compete with Backup Exec on these levels?

At the moment BE does not do exactly what you want it to do for your environment - there is not much I can do at this stage to resolve that issue for you

However, what you are suggesting would be an excellent add-on to BE - post an enhancement request to http://enhancement.veritas.com.

[Ross_Smith]

John, you're right, Winzip doesn't compete on all of those levels: that's why I bought BackupExec. The reason for my current frustration is finding that I may now have to revert back to Winzip...

However, if the advanced backup is per media server then that may be worth me taking a look. Do you know if I'm able to get a 30 day evaluation of that?

I will suggest the Winzip enhancement, but in my experience enhancement suggestions just seem to disappear into a black hole. Is there no way to suggest enhancements on an open forum for discussion among the user base? Surely allowing users to vote on the enhancements would give veritas a much clearer idea of what people really want.

Ross

[John_Chisari]

Ross

The Advanced Disk Based backup option is part of a normal BE evaluation - you will get 60 days eval of it. You may just need to install onto a different server or workstation to test it out if you don't want to take out the licenses for your current BE installation.

All enhancement requests do looked at - it will obviously take a while for something new to come out - and also the amount of requests for the same feature does come into consideration.

I spose if anyone else who is reading this thread would like this feature, post to http://enhancement.veritas.com.

I do look forward to seeing how you fare with Synthetic backups and if it can work in your environment. I have spoken to a few customers since BE 10 came out, Synthetic backups have made their lives alot easier.

John

[Mark_Greenberg]

Haha, this is kind of funny because the reason I came to these forums in the first place was to figure out how to get Synthetic backups to work. I just happened to also see Ross' problem and piped in.

Unfortunately, from what I can tell synthetic backups don't actually work. It seems everyone posting about them has problems that are never solved, and in the end you are refered to phone support you have to pay for.

Ross, before you try synthetic backups, be sure to try it on a seperate test server, because whatever bug there is in that feature results in the backup exec engine service crashing, which means none of your backups run. So don't evaluate it on your live backup server, set up a whole seperate test server for it.

If you do get it to work, let me know please how you have your backup server configured.

[Ross_Smith]

Yeah, I was thinking that when synthetic backups were suggested... I've seen quite a few threads about them.

However, I do have a fully paid up support contract, and I'm no stranger to Veritas Support :-). I'm actually in the process of re-formatting my backup server at the moment in the hope of getting BackupExec to work reliably on a clean install of V10 as opposed to an upgrade from 9.1.

I'm going to give that server a week to make sure our regular backups are working reliably and then I'll try synthetic backups. Hopefully they'll work well on a completely clean install. I will keep you posted.

Ross