Showing results for 
Search instead for 
Did you mean: 
Level 6

Data management landscape

Many organizations are struggling with the massive changes in data storage requirements that have transpired over the last decade. The almost exponential growth of business-critical data from email, e-commerce, and electronic systems shows no sign of decreasing. With relatively new data types such as voice and video now in use, enterprise storage administrators will soon have to manage petabytes of data.

According to a recent study by the School of Information Management and Systems at the University of California at Berkeley, the world produces between 1 and 2 exabytes of unique information per year, which is roughly 250 megabytes for every man, woman, and child on earth. An exabyte is a billion gigabytes and printed documents of all kinds comprise only .003 percent of the total. Magnetic storage is by far the largest medium for storing information and is the most rapidly growing, with shipped hard-drive capacity doubling every year. Magnetic storage is becoming the universal medium for information storage.

The annual growth rate of corporate reference data is estimated to be 60 percent. Traditional ”weekly full, daily incremental” backup approaches are ill suited to cope with this situation: companies should work 60 percent more efficiently to prevent costs from increasing, which is not very likely to happen with the current systems. As the business world has moved to a 24-hour, 7-day-a-week working cycle, the notion of overnight “downtime” for maintenance and backup is less feasible. Symantec has developed software to help companies manage data integrity more efficiently and meet today’s standards for data protection.

The rise in the volume of data has seen a corresponding tightening of corporate governance and legal procedures surrounding the retention and availability of data. According to one large storage vendor, there are over 4,000 major regulations that apply to information-keeping worldwide. The United States has the most, with federal statutes such as the Health Insurance Portability and Accountability Act (HIPAA) that covers medical records, and the Food and Drug Administration Section 21 rules that carry heavy fines for noncompliance with data-retention rules. The most commonly quoted U.S. regulation is the Sarbanes-Oxley Act of 2002, which was brought into force after Andersen employees shredded important documents in the wake of the Enron scandal. The Securities and Exchange Commission (SEC) also has extensive rules governing data retention, with heavy fines and even jail sentences for executives in cases of noncompliance.

The U.S. National Archives and Records Administration (NARA) and the United Kingdom's Public Records Office are two examples of bodies whose entire business is focused on ensuring records are maintained correctly and effectively. In fact, most developed countries have similar governmental structures to ensure that data is archived and released within a legal framework. These guidelines, in turn, must be adhered to by other governmental agencies such as the Social Security Administration and the Internal Revenue Service, as well as banks and building societies.

In Europe, draft European Union (EU) legislation being formally ratified now will force telecom companies and Internet service providers (ISPs) to retain information on their customers’ logs of phone calls or e-mail and Internet connections beyond the one- or two-month period the information is normally held for billing purposes. The period could be up to a year; this cange is intended to assist police and fraud investigations.

Data and risk evolution

Conventional data management and protection techniques have not kept pace with the increasingly complex nature of today’s data processing and topology. Many IT organizations still use the ”weekly full, daily incremental” backup technique employed since the 1950s. In the past half century, topologies have evolved from centralized homogeneous platforms to heterogeneous networks of distributed and mobile systems with multiple storage tiers; annual data growth has increased from 20–35 percent to 80–100 percent; retention periods have increased from weeks to decades; and usage patterns have evolved from transaction to transaction and reference. Studies indicate approximately 60 to 80 percent of this growth is fuelled by reference data. Reference data describes information with access requirements measured in seconds to minutes while transaction data describes information with access requirements in milliseconds.

Traditional data-protection and management approaches fail. Ernst & Young’s Fabric of Risk study determined that approximately 36 percent of the executives from the top 1,000 publicly traded companies believed their companies would cease operations due to inadequate protection, while 59 percent placed their risk as moderate to high. The increased dependence upon networked and mobile data, combined with theft and vulnerability to viruses, exacerbates this risk. New technologies such as storage area networks (SANs) aid physical storage management, but affordable data-protection and management solutions have been elusive.

To read the complete article, download the PDF.

Version history
Last update:
‎02-27-2009 03:04 PM
Updated by: