Classically, there have been few viable solutions for Long-Term Retention (LTR) of digital data. To date, the most popular is tape media—either stored in the datacenter or in an offsite location. Tape storage, while a low-cost LTR option, is often not immediately accessible given physical constraints; tape media generally lasts 20-30 years, though the hardware required to read the media is another story. And maintaining a tape infrastructure requires a data center footprint, administrative effort and time, and regular capital investments to refresh the hardware.
With the rise of cloud service providers, new options are available that are changing the landscape of how data is handled. Remember when you had to seek a CD or DVD out of that stack of disks when you wanted to listen to a certain song or movie? Or when you had to copy music or video to a media device or phone? The modern way consumers access personal data has changed significantly in recent years. Now, we can have nearly any piece of media delivered almost immediately. Logically, the enterprise is making the same shift, even with data we need to retain for the long term.
Migrating from legacy tape-centric architecture to a disk-only solution is enabled by deduplication technology, which helps to manage storage costs. The same is possible with cloud-based storage. AWS has added Glacier and Deep Archive storage tiers to their roster of object storage offerings. These tiers are low-cost offerings intended to be competitive, long-term storage targets. As IT teams take on today’s cloud-based options, Veritas NetBackup can bridge the gap between legacy tape solutions and the cloud.
Not only can Veritas NetBackup write to object storage, but it can also optimize that data for storage prior to writing, allowing customers to reduce costs even further. The NetBackup CloudCatalyst feature is an object storage-based deduplication technology that is fully compatible with the traditional MSDP deduplication use by NetBackup. With this technology, users can realize storage savings, as well as true end-to-end deduplication no matter the source or number of data storage locations. Veritas can help save money with LTR in other ways, too. Veritas Information Studio can map and provide insights to data to help identify data ideal for long-term storage. Using these tools, Veritas can help customers get the maximum value out of AWS storage solutions.
There are several ways to move data into and around AWS storage solutions, though let’s focus on using NetBackup CloudCatalyst and S3, Glacier, or Glacier Deep Archive as a replacement for tape.
Tape replacement with CloudCatalyst
A common backup scenario is to keep new data on disk-based storage for fast recovery. Older copies of data are migrated to tape for long-term storage and disaster recovery, and short-term disk copies are deduplicated to save on local storage costs. When writing out to tape, data must be rehydrated—a time-consuming and costly process. With CloudCatalyst as the target, we no longer need to rehydrate data. With CloudCatalyst, optimized copies can be replicated from multiple locations to a CloudCatalyst server, which then writes deduplicated data directly to object storage, including low-cost Glacier and Glacier Deep Archive tiers. If those tiers are already providing cost savings, imagine the savings achieved by deduplicating the data before it’s stored. That data will stay deduplicated no matter where it goes; never rehydrate unless it’s to restore data.
In the image above, a CloudCatalyst node has been added to a step traditionally managed in a tape library. There are a few advantages in this case that wouldn’t be available with tape. As stated before, when using CloudCatalyst for LTR, data sent from MSDP will not need to be rehydrated, providing significant performance and storage savings. And because CloudCatalyst is a standard media server leveraging the deduplication engine in MSDP, the data format is compatible between the different targets. This means data stored in cloud object storage by CloudCatalyst is highly portable. Copies can be replicated to and from CloudCatalyst to any other compatible storage server. These server types include other CloudCatalyst servers, traditional MSDP media severs, and even an Access appliance. With features like Automated Image Replication (AIR) and Accelerator, NetBackup can deliver true end-to-end deduplication, making storage utilization optimized for the entire lifecycle of the data.
Snowball seeding with CloudCatalyst
Deduplication and compression technologies dramatically reduce the amount of storage needed, and are especially useful in backup, managing multiple copies of data. Even with the benefits of deduplication, the first pass of data that acts as the “seed” for the rest of the data to be protected can consume notable space. To cope with the seeding scenario, Veritas has ensured support for AWS Snowball devices. Snowball and Snowball Edge devices are ruggedized servers, used to import and export large amounts of data from AWS.
The way Snowball works is pretty straightforward. In the example above, a job is created in AWS to deliver a Snowball Edge, already preloaded with the buckets or compute needed for the job. The Snowball Edge arrives a few days later and can be configured as a S3 target for a CloudCatalyst server. Once the initial “seeding” job is run, the Edge can be shipped back to Amazon where the data will be imported into AWS. When the import job is complete, the CloudCatalyst server can be reconnected to the S3 bucket in AWS where it can resume jobs. From this point on, all data sent to AWS will be completely optimized for efficient network transport and reduced storage consumption.
With Veritas NetBackup and Amazon’s S3 storage solutions, aging tape infrastructures can be replaced with new cloud-based solutions, leveraging modern technologies to reduce storage costs, eliminate network bottlenecks, and provide a streamlined and automated data protection lifecycle that will protect data for as long as is needed. NetBackup’s software-defined nature makes it future-ready, and users can run NetBackup on premises with NetBackup Appliances, with Build Your Own (BYO) servers, or as virtual instances on your hypervisor of choice. You can even run NetBackup out of the Amazon AWS Marketplace. And remember, NetBackup services will work seamlessly with each other no matter where they are running, providing the ultimate flexibility.
For more details on Veritas solution for LTR protection with AWS, check out this fantastic white paper here.
