I can vividly remember the late nineties when I proudly purchased the largest hard drive capacity I could find, which boasted 30 GB! I remember thinking that I could never possibly fill it all. Now, 20 years later, you can easily purchase a TB thumb drive at your neighborhood convenience store, and it’s hard to even wrap your head around the numbers shared for data created daily. Experts estimate that over 44,000,000,000,000,000,000 KB or 44 zettabytes[1] of data exist in the digital universe today, with about two-thirds of that data stored in the cloud—and it’s all growing exponentially! Ninety percent was created in the last few years alone, which means that now more than ever, your company’s ability to handle, store, navigate and restore terabytes, petabytes or even exabytes of data is critical to your success. Manual is no longer an option! Zettabytes are coming. So, what’s your data strategy?
Without a data strategy that includes intelligent automation, compliance, performance, resiliency, protection, and differentiation between production and secondary data across physical, virtual, and all major cloud platforms, organizations may find themselves feeling like a gridlocked city. Like cars navigating city roadways, data can be directed via policy through the various stages of its lifecycle. This concept has existed under many names, such as data lifecycle management or the data journey. The bottom line is that having a framework for directing and navigating data can make the difference between keeping your business running and getting buried under ever-increasing storage costs.
To put that in perspective, in 2005, Apple introduced the iPod Shuffle, the companies tiniest device yet; Intel and AMD launched their first versions of a dual-core processor; Google Maps launched and three PayPal employees quit their jobs and started a new company for video-sharing called YouTube! While the world was busy taking advantage of significant advancements in data mobility and media options—along with smaller, faster, and more powerful technology thanks to Moore’s Law—our engineering teams were busy creating SLPs.
Sonya: Hi Dennis and Tom. Thank you both for agreeing to chat with me today. I’m looking forward to hearing your insights and learning more about the origins of SLPs. Dennis let’s start with you. When did you start working for Veritas?
Dennis: I joined the Veritas team in 2005. It was an exciting time to join the team because NetBackup was preparing to release what is today known as OST, and the team was formally tasked with finding a better way for disk staging to work with newer technologies.
Sonya: Sounds like an exciting time to work for Veritas indeed! And 15 years is tremendous. Tom, when you started working at Veritas, what project did you work on?
Tom: I started working in May of 1999 for the media manager team. I primarily worked on the tape library control software. In those days, backup media was always tape.
Sonya: Impressive. After 21 years, you must have really seen NetBackup evolve! I understand the early 2000s were a time of significant innovation for our industry, especially in a disk-based backup. Tom, can you share more insight into what was happening in the disk industry leading up to 2005?
Tom: Sure. During the early 2000s, many disk storage manufacturers had come up with some really optimized ways to improve the density or the storage consumption on their disk storage—and sell more disk. They utilized backup storage, which became known as Virtual Tape Libraries or VTLs, which helped provide a higher level of integration for disk-based backups.
The interesting story came next. The best practice we still use in the industry today evolved out of marketing campaigns developed by the disk storage industry during this time. Disk-to-Disk-to-Tape or D2D2T is a backup strategy for storing the same data on disk and tape. The primary disk runs the app. The first backup goes to disk in the form of a VTL and then the data is migrated to tape for the long-term copy. Disk staging was our story at the time, but we knew we needed a better way to manage deduplication long term. How Disk Staging Storage Units work is they monitor the disk storage until it reaches a predetermined "high water mark" and then it expires and removes the oldest images. Because deduplicating storage has a much more involved data persistence paradigm, the notion of freeing up space immediately was no longer predictable, and Storage Lifecycle Policies were created to manage the additional complexity of modern disk storage systems.
Sonya: So marketing played a key role by influencing the technology. As a marketing professional, I enjoy hearing that!
Tom: Well then, here’s one more for you. There was another key marketing campaign happening in the industry at that time—Data Lifecycle Management or DLM. Veritas created its own version of DLM called the Storage Service Plan. DLM automatically migrates your images from fast recovery disk to a cheap disk-like VTL and then to tape.
Sonya: Is this also when data deduplication technology become implemented?
Tom: Yes, that’s the last crucial part of SLPs. OST or Open Storage Technology was born out of a partnership with NetApp and an acquisition of a dedupe storage company in 2005 to take advantage of implementing their new deduplication technology. So, with the integration of these three—D2D2T, Data Lifecycle Management, and OST—SLPs were invented.
Sonya: Thanks Tom, what a nice history lesson. Would you say that is what makes SLPs so impactful?
Tom: Well I would say that the clever part of SLPs is rooted in one more history lesson. During this time, the tape industry was also having major breakthroughs with the Fiber channel, significantly improving its throughput. Data could stream to tape at 10x or 50x the rate that you could write to disk. That was keeping VTLs at bay. But it also created a problem for these tape companies in the industry: They couldn’t keep their drives spinning. We recognized this as an opportunity right away. We created our SLP solution system to use spare spin cycles in the system to conduct secondary operations. In other words, it didn’t cost you more money to include this capability and it maximized space. This was huge.
SLPs also developed interesting technology around capacity management and various retention capabilities like keep for a fixed period of times or cascade copy, so as soon as the next copy was made, the old one would be deleted, offered you a lot of power for managing how much storage you were consuming.
Sonya: Wow, so now backups and restores were faster and more reliable, deduplication reduced storage requirements significantly and disk costs were close to that of tape. Game-changing. In what version of NetBackup did we launch this technology?
Dennis: NetBackup 6.5 was a big disk Data Protection‒focused release and it officially launched SLPs. NetBackup already had some old disk-based features that included Vault and Disk Staging, but we recognized neither one was ideal for working with new technologies. Along with the noted advancements in the industry and pressure from disk array partners for a better solution, that prompted a formal engineering lead initiative to solve this problem by 6.5.
Sonya: Can you share more about what you meant by pressure from disk array partners?
Dennis: A conscious decision was made early on to keep NetBackup vendor-agnostic. This approach was contrary to the industry at the time and sometimes a hot topic within the team. We worked closely with NetApp to create a standard interface to back up to disk, OST. We grew quickly to have over 30 disk array vending partners. For the partners, the beauty of the open storage interface was that they could do as much or as little as they wanted to add value on top of their disk. We continually added value to the OST partnership through new feature content, differentiating for us and the storage providers, which also grew the capabilities of our SLPs. Safe to say that these essential storage partnerships were the fundamental building blocks of the SLP management layer that made them all work well together.
Tom: Yes, it was a really fun time. They brought feature ideas to us and we did to them. Clearly their products didn’t sell without ours, so they were highly motivated to drive the innovation at the time. We got a better solution through our relationship. They made us better. These partnerships were vital.
Sonya: Dennis, you shared that NetBackup being vendor agnostic has been a hot topic. Can you share more?
Dennis: The dichotomy of trying to balance the agnostic approach with doing things that would make our own storage solution more attractive or sell more at Veritas was deeply debated. Internally, there was a large amount of discussion and introspection. Especially during 2005, we had just bought a company specializing in deduplication technology, so there was pressure to make our own storage solution more attractive. Fortunately, many understood the value of the relationships we’d formed at the time. Although challenged a few times over the years, staying vendor agnostic is still a priority. Now that we’re working with public cloud providers today, it has proven to be vital to our success again.
Sonya: How did SLP technology evolve after NetBackup 6.5?
Tom: By 7.0, the team decoupled the management of snapshots independent from backups, using our SLP engine. This was a crucial change, especially for those protecting large databases. We also added replication, indexing from snapshots, replication of snapshots, the ability to set windows and priorities, and developed our own deduplication.
Sonya: Can you elaborate on what you mean by windows?
Dennis: After significant data growth, there began to be an issue with too many read/writes happening at the same time on a backup server. It was slowing everything down. So, one of the steps we took in 7.6 was SLP windows, which scheduled operations at different times. The technology acts as a traffic cop, directing backups to be performed at night and duplications and replications during the day. Everything is timed so operations run more smoothly.
Sonya: What would you say are the key differentiators of SLPs?
Tom: Since the original release we’ve obviously extended SLP technology to other operations beyond duplication. A key differentiator has been our strategy to use downtime. Less-important tasks are offloaded to a different time, traditionally your downtime, so you can maker fuller use of your hardware. This approach has helped our customers find greater efficiencies, save time, and maximize their ROI. I think the key values of SLPs are its reusability and its ease of modification. It was built on solid technology. Over the years, our competitors have used different approaches, and some even had additional operations that had to go into each equivalent of their policy and define additional operations. We were the first in the industry to formalize a reusable plan.
Dennis: I would say that among our customer base, there isn’t a customer around that doesn’t use SLPS. Most of them use SLPs for all their backups. SLPs make everything so easy.
Tom: Yes, maybe even too easy! Setup is even a breeze. Within a minute of working with an SLP you can have that exact same methodology planned out across in your entire environment.
Dennis: Basically, as soon as that backup job is done, the backup image has been created. Everything that happens to that image after the copy of the user data is under the control of an SLP. The automation is powerful. Our customers can set it and forget it with an SLP. They set the retention period they want, set the area for where they want that data to live, and then the SLPS will follow that prescriptively—without customers having to do anything manually.
Tom: Today, being vendor agnostic and software-defined means our customers can scale quickly and take advantage of any cloud providers they want—on-prem, third party, and so on.
Sonya: You touched on my next question. How relevant are SLPs today?
Tom: SLPs are crucial today, actually now more than ever. There was a time when a single backup copy was sufficient. You might have backed up your files just in case something happened like maybe you accidentally delete a file and need to recover it. That’s no longer the case. Now, not only is data exploding at exponential growth rates and stored in many different media types, but you’re now backing up your data for other reasons such as regulatory compliance, disaster recovery, ransomware resilience, or business continuity. So, the application of that backup data has grown significantly and the need to manage multiple copies of it has also grown significantly.
Dennis: SLPs are vital. As data grows exponentially into the levels of petabytes, SLPs give our customers significant savings in time, not to mention peace of mind.
Tom: Did you know that Dennis is like the SLP wizard of Veritas?
Dennis: Well, the bulk of my time at Veritas has been working on SLPs, and it’s one of the primary reasons that I’ve stayed so long. It’s powerful technology.
_______
A big thank you to both Dennis and Tom for taking the time to share their insights into the rich history of SLPs. I enjoyed the history lesson of disk-based backup storage technology advancements during those early years and how it evolved into SLPs released in NetBackup 6.5.
Despite all that has changed over the past two decades, one thing hasn’t: A reliable data strategy is paramount when organizations are faced with disaster recovery, pandemics, regulatory compliance, exponential data growth, and threats like ransomware. Data loss is not and should not be an option. At Veritas, we don’t do enough to highlight the technologies that really make us great. Well here’s one, SLPs! Why? Veritas intelligently created scalable, intelligent, and dependable technologies alongside key partnerships to build vendor-agnostic solutions that are just as relevant today, as our customers move to the cloud, as they were back in 2005. Unlike our competitors, Veritas fully understands various customers’ data journeys. NetBackup SLPs are easy to use, easy to set up, easy to change, and easy to scale. Our 30-year legacy brings with it a wealth of experience for intelligent solutions, allowing our customers peace of mind that their most important asset—data—is well protected, efficiently, and at scale. It will only get better from here. When data hits quantities in the zettabytes and yottabytes, our customers will be ready!
[1] https://www.forbes.com/sites/gilpress/2016/08/05/iot-mid-year-update-from-idc-and-other-research-fir...
