A transportation company was testing Symantec Backup Exec 11d for Microsoft Exchange Server to see if it would make the company's backup and restore operations more efficient. The plan was to use the new Granular Recovery Technology (GRT) to do a single-pass backup that could recover individual messages without restoring the entire database.
But things kept going wrong with the restores. Parts of email messages were missing and some files attached to emails were no longer attached when they were restored. The data was still in storage and not permanently lost, but the company was not achieving the ease of restore Backup Exec 11d promised.
A Symantec QA engineer dispatched to the customer's data center and evaluated the problem. Upon his return, he contacted Bill, a Symantec tech support engineer with nearly a decade's experience with Backup Exec. Bill and the QA engineer obtained a copy of the customer's database.
Testing is different from live
What was going wrong? "Data in a live Exchange environment is often subjected to other applications modifying it. Spam filters, antivirus and custom workflow applications can all alter the properties of e-mail messages" Bill says. "This is stuff that's very difficult for us to duplicate in a laboratory. Until we actually got the product out in the wild and saw these things happen, we couldn't have duplicated them in the lab."
Now that Bill and his colleagues had an actual, live data set to work from, they set about resolving the issues. "We got a copy of their database, mounted it in our test environment, and went through the process of backing up and trying to restore those items to find the specific roadblocks. Then our developers found ways in the code to work around them." The process took a few months to complete, but all the glitches were fixed to the customer's satisfaction.
"The customer had a lot of technologies that were difficult to configure," Bill recalls. "I happened to have expertise with Microsoft Exchange Server and with Oracle Server in a mixed environment with Windows and Linux, all of which they had." Subsequently, Bill, along with the original QA engineer, was dispatched to the customer's site to configure Backup Exec with these various resources. They also implemented additional patches to make sure BE was optimized for this environment. The customer was ultimately able to use the Exchange GRT to achieve the desired result of a much shorter backup window for Exchange data.
Sharing the fix
Once Bill and his team had resolved the company's problems, they offered the fix to all customers as Hotfix 14 for Backup Exec 11d, Build 7170. "If there are a lot of different filters or properties on a particular email or object, we can formulate a list to look for those conditions. And then when we display that object for the purpose of a restore, we can make sure we're going to have all those properties defined and attached to that message or object correctly," Bill explains.
Meanwhile, the company decided to go ahead with its Backup Exec 11d implementation, and is already seeing benefits. "When they were backing up the database and the individual mailboxes, it literally took all weekend," Bill says. "It would actually go outside the restore window, into production time, where people trying to log in to the system would find it very slow.
"Now that same job takes two or three hours. And they can still restore individual things, just like they could from the job that took all weekend."