Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- VOX
- VOX Knowledge Base
- Community Insights Knowledge Base
- Articles
- When is Data Relevant? Part I – Just retaining the...
Options
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
Options
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
on 06-11-2009 09:42 AM
Understanding the importance of data is not a new phenomenon, in fact, it is as old as the hills. But understanding the way that we deal with electronic data and the management of its life-cycle is a relatively new discipline that has been forced upon us through the advent of new data compliance legislation, Governmental directives and quality codes of conduct across all walks of business life.
Legislation worldwide has evolved to cover new technologies, focusing on the issue of data retention. Now businesses, and specifically CIOs and IT Managers are facing a series of changes in the way that organisations need to deal with, and manage, their electronic data. What makes compliance so complicated is that no matter which set of regulations or vertical industry you look at, there are numerous stakeholders which involve System Administrators, CIOs, CFOs , operations departments and HR departments all working together.
IDC research consistently reveals that Data Centre storage requirements are increasing 50-100% per year for the next several years. Gartner Group advises that 50%-80% of corporate data is unstructured.
Data lifecycle management is real. As IT desperately tries to keep up with the terabytes of data users capture for which they are responsible there will come a point where the amount of data becomes irrelevant – unless the storage industry enables IT to manage, store, find, retrieve and delete through management tools and innovation, making sure IT keeps data manageable and relevant to the end user.
To make things slightly more complicated CIOs and IT Managers or Directors are tasked with compliance issues from the complete range of internal customers including: Legal, Finance, Operations, Quality Management, HR, Facilities, Sales/Marketing and these requests present IT professionals with significant resource issues relating to: personnel, budgets, additional hardware/software requirements, as well as bandwidth and training.
Not only do IT departments have to pull the proverbial rabbit from the hat, but they are faced with trying to do so without an IT infrastructure to cope with the increasing demands relating to compliance and data life cycle management. Given the prevalence of data and security compliance related legislation it seems incredible that so many of these IT departments still do not have budgets allocated specifically for compliance issues.
In Europe, Government, Financial, and many more commercial organisations are faced with a wide ranging legislative and regulatory requirements, as well as best practices and quality audit or best practices requirements. These include a huge range of requirements which include:
Data Protection, Privacy and Electronic Communications, Investigatory Powers, Computer Misuse, Criminal Justice Anti-money laundering, Anti-terrorism legislation, Ecommerce Regulations, Human Rights, Copyright Patents and Designs, Freedom of Information, Financial Services rules, Financial Reporting Standards, Operational Risk Systems and Controls guidelines, ISO, EU Directives, etc.
The real issue is that directors or board of trustees now have an obligation, have a duty to act with reasonable skill, care and diligence to safeguard the shareholder’s or company assets. A matter that was dealt with quite by normal routine operational activity has now reared its head and threatened the board room.
Many records will need to be retained for specific periods: from 3 months to 99 years. But simply retaining information is not the point; easily retrieving the information is the issue.
We have, for some time talked about the relevance of storing and managing data with the Investigatory and Anti-Terrorism Legislation the demand for increased security has meant an increased pressure on the airline and telecom industries. The underlying fact is that data retention and ease of retrieval has become important more than simply for Disaster Recovery as governments move towards increased regulations across industry sectors. It has always been commercially foolish not to look after one’s data – now it’s a criminal offence!
Simply, our ability to cope with these new regulations would not be possible without new technologies, such as data, or information, lifecycle management, storage networking and storage resource management (SRM). These new conciliatory requirements, together with the move to consolidate IT facilities and cut costs will continue to compel companies to reshape their storage infrastructures and start to look more carefully at the way administrators plan and implement storage capacity examining the life cycle of data and how it is effectively managed as it ages.
The retention of data has become more important than ever. New legislation introduced over the last two years has placed increasing demands on all businesses, public or private, that intend, or currently have, any form of electronic record management system – which basically means everybody. Data retention can be defined as pretty much anything, whether storage of mobile communications traffic, location and subscriber data, as well as web data capture or good old fashion banking details.
What most of the legislation we are now facing requires is that this electronic data is available for subsequent retrieval and use as evidence or intelligence by law enforcement and security agencies as well as financial auditors, quality auditors and data protection agencies. Well, that seems simple enough. All we have to do is make sure we store all this stuff and can retrieve it when requested. Or is it that easy?
Recent history around data protection, retention and retrieval is splattered with business faux pas and major events pretty much proves the point that companies need to have effective document retention policies in place. Even if we take a mercenary view of data protection, not only does a decent data retention policy avoid vast fees in future litigation, but utilising intelligent software can save storage space and costs.
But it’s not just data that organisations have to make available to the general public as part of the data protection laws, or freedom of information legislation that we all have to worry about. As if wasn’t a big enough head ache for, for example, the governmental requirement in many countries is for local government to store data in a tamper proof storage system, or the retention of trade transaction messages stored unaltered up to 99 years, no, there is a myriad of laws, regulations, legislation, codes of conduct, and audit requirements that will have to be adhered to by all organisations, and not necessarily just in their native country.
So what are the requirements? Well, in terms of the most recent of the US and European directives, the bottom line is that:
• All personal data must be connected to some definite term, beyond which storage and processing is not allowed. In other words, if that term is 3 years, then the data cannot be altered in any way for the remainder of the term that it has to be kept, say, a further 7 years.
• All persons about whom data is held, or archived, must be made aware of what is contained in that electronic archive and have access to their personal data
• Email and instant messaging must be retained in its entirety and must be managed and treated in the same way as personal data
• All data held must be correct
• All data held must be kept secure with appropriate access restrictions confined to those with legitimate reasons for having access to it.
Which all in all makes data retention an exact science – all organisations have to get this right. Needless to say that if legislation says you have to comply with x then every so often someone is going to want to make sure an organisation has done right by x. Failure to do so in Europe will mean criminal sentences for CEOs and board members, heavy company fines, long drawn-out litigation and businesses going under. It’s like Y2K all over again, but a real threat and on-going forever.
So, the first thing an organisation needs to do is look at its current electronic data/document retention policies and abilities and try to measure the gap between its ability to comply and the compliancy requirement. Fundamentally every organisation needs to consider its data retention and retrieval policies on a global basis. It is not just about whether such and such legislation is required in its own country, nor is it whether or not an organisation wants to do business in this or that country. Legislation around this subject will inevitably become global in nature, so even if storing data is currently not legislation in say, Latvia, it won’t be long before it is.
Most countries have legislation that gives individual access to data, sets up data management practices, guarantees security of data, ensures that individuals can request copies of their own data and gives a right of privacy to individuals - so businesses should start thinking along these lines. Most countries do not have specific retention periods set down by its own limitation rules. However, the statutory limitation periods are relevant because they indicate the length of time documents will be required to bring or defend proceedings however, if, as in many countries, there are rules that certain documents should always be kept in original form and never destroyed then this point becomes irrelevant. Many countries have law on specific vertical markets, or specific company law, or regulatory requirements pertaining to certain company documents that must be retained for fixed periods.
Now, although organisations could approach the problem with a wide sweeping statement that all data must be kept forever, not all data is equal – some is more equal than others. And although by addressing data retention legislation, which suggests that all data is important to all organisations, forever, there is less inherent value to a business in data as it gets older. The idea behind data information life cycle management, therefore, is that data has different values at different moments in time. When an email first arrives in your in-box the information may be urgent or indeed vital for securing a business deal or transaction. After 2 or 3 days the longer the information held in that email may be of little significance and so it’s importance to the recipient becomes less and less. However, that information might have value to other organisations or individuals even if they're no longer useful to the original recipient.
The life cycle of Data begins with a business need for acquiring data. Delivering critical business information into the hands of the people who need to make both strategic and tactical decisions every day is an important trend in today’s market. Active data is referenced on a regular basis during day-to-day business operations. Over time, this data loses its importance and is accessed less often, gradually losing its business value, becoming inactive and eventually ending with its archival or disuse. But even in its archived state, data still has some inherent value to a business. The simple, but vital principle, that all data moves through life-cycle phases, is the starting point for improving data management. By understanding how data is used and how long it must be readily available, companies can develop a strategy, by using policy based management, to map usage patterns to the optimal storage media, thereby minimising the total cost of storing data over its life.
Active Data
Data becomes active as soon as it is captured by an organisation or becomes of business interest to it. This is usually at entry point into the organisation. This data must be accessible easily, swiftly to the business user in order for organisations to run efficient businesses. The same principle applies when data is stored in relational databases, although managing and storing relational data is compounded by complexities inherent in data relationships. Relational databases are a major consumer of storage and are also among the most difficult to manage because they are accessed on a regular basis. Without the ability to manage relational data effectively, relative to its use and storage requirements, runaway database growth will result in increased operational costs, poor performance, and limited availability for the applications that rely on these databases. The ideal solution is to manage data stored in relational databases as part of an overall enterprise data management solution. Data deteriorates at around 10% per month, in other words, for any single record item, 10% will alter within a month. So unless a record is in constant use (this could be customer records for example), its value to the business will deteriorate rapidly and will cease to be active.
Inactive Data
Once data ceases to be active it becomes inactive data. This data is no longer business critical and can be migrated to cheaper storage. Inactive data is old files no longer accessed, files that may be of use in the future but are no longer accessed on a regular basis. Inactive data can be from 1 year to 3 years old. It is still necessary to be able to access and retrieve this type of data for compliance purposes but is likely to have less inherent value to the business.
Archived Data
Archived data is information that is no longer an immediate use to the business. Prior to the mid-nineties, most organisations archived data in Microfilms and tape back-ups. There are now technologies for data archival such as Hierarchical Storage Management, Data Lifecycle Management and Information Lifecycle Management, as well as the supporting infrastructure and media technologies such a NAS, SAN, Serial ATA, etc….. These storage systems can maintain referential integrity and business context. Furthermore backup vaulting can help to manage the process of storing and retrieving data as well as enabling a dramatic reduction in backup and restore times as well as media utilisation as data that is captured in the archive, no longer needs to be backed up regularly.
Legislation worldwide has evolved to cover new technologies, focusing on the issue of data retention. Now businesses, and specifically CIOs and IT Managers are facing a series of changes in the way that organisations need to deal with, and manage, their electronic data. What makes compliance so complicated is that no matter which set of regulations or vertical industry you look at, there are numerous stakeholders which involve System Administrators, CIOs, CFOs , operations departments and HR departments all working together.
IDC research consistently reveals that Data Centre storage requirements are increasing 50-100% per year for the next several years. Gartner Group advises that 50%-80% of corporate data is unstructured.
Data lifecycle management is real. As IT desperately tries to keep up with the terabytes of data users capture for which they are responsible there will come a point where the amount of data becomes irrelevant – unless the storage industry enables IT to manage, store, find, retrieve and delete through management tools and innovation, making sure IT keeps data manageable and relevant to the end user.
To make things slightly more complicated CIOs and IT Managers or Directors are tasked with compliance issues from the complete range of internal customers including: Legal, Finance, Operations, Quality Management, HR, Facilities, Sales/Marketing and these requests present IT professionals with significant resource issues relating to: personnel, budgets, additional hardware/software requirements, as well as bandwidth and training.
Not only do IT departments have to pull the proverbial rabbit from the hat, but they are faced with trying to do so without an IT infrastructure to cope with the increasing demands relating to compliance and data life cycle management. Given the prevalence of data and security compliance related legislation it seems incredible that so many of these IT departments still do not have budgets allocated specifically for compliance issues.
In Europe, Government, Financial, and many more commercial organisations are faced with a wide ranging legislative and regulatory requirements, as well as best practices and quality audit or best practices requirements. These include a huge range of requirements which include:
Data Protection, Privacy and Electronic Communications, Investigatory Powers, Computer Misuse, Criminal Justice Anti-money laundering, Anti-terrorism legislation, Ecommerce Regulations, Human Rights, Copyright Patents and Designs, Freedom of Information, Financial Services rules, Financial Reporting Standards, Operational Risk Systems and Controls guidelines, ISO, EU Directives, etc.
The real issue is that directors or board of trustees now have an obligation, have a duty to act with reasonable skill, care and diligence to safeguard the shareholder’s or company assets. A matter that was dealt with quite by normal routine operational activity has now reared its head and threatened the board room.
Many records will need to be retained for specific periods: from 3 months to 99 years. But simply retaining information is not the point; easily retrieving the information is the issue.
We have, for some time talked about the relevance of storing and managing data with the Investigatory and Anti-Terrorism Legislation the demand for increased security has meant an increased pressure on the airline and telecom industries. The underlying fact is that data retention and ease of retrieval has become important more than simply for Disaster Recovery as governments move towards increased regulations across industry sectors. It has always been commercially foolish not to look after one’s data – now it’s a criminal offence!
Simply, our ability to cope with these new regulations would not be possible without new technologies, such as data, or information, lifecycle management, storage networking and storage resource management (SRM). These new conciliatory requirements, together with the move to consolidate IT facilities and cut costs will continue to compel companies to reshape their storage infrastructures and start to look more carefully at the way administrators plan and implement storage capacity examining the life cycle of data and how it is effectively managed as it ages.
The retention of data has become more important than ever. New legislation introduced over the last two years has placed increasing demands on all businesses, public or private, that intend, or currently have, any form of electronic record management system – which basically means everybody. Data retention can be defined as pretty much anything, whether storage of mobile communications traffic, location and subscriber data, as well as web data capture or good old fashion banking details.
What most of the legislation we are now facing requires is that this electronic data is available for subsequent retrieval and use as evidence or intelligence by law enforcement and security agencies as well as financial auditors, quality auditors and data protection agencies. Well, that seems simple enough. All we have to do is make sure we store all this stuff and can retrieve it when requested. Or is it that easy?
Recent history around data protection, retention and retrieval is splattered with business faux pas and major events pretty much proves the point that companies need to have effective document retention policies in place. Even if we take a mercenary view of data protection, not only does a decent data retention policy avoid vast fees in future litigation, but utilising intelligent software can save storage space and costs.
But it’s not just data that organisations have to make available to the general public as part of the data protection laws, or freedom of information legislation that we all have to worry about. As if wasn’t a big enough head ache for, for example, the governmental requirement in many countries is for local government to store data in a tamper proof storage system, or the retention of trade transaction messages stored unaltered up to 99 years, no, there is a myriad of laws, regulations, legislation, codes of conduct, and audit requirements that will have to be adhered to by all organisations, and not necessarily just in their native country.
So what are the requirements? Well, in terms of the most recent of the US and European directives, the bottom line is that:
• All personal data must be connected to some definite term, beyond which storage and processing is not allowed. In other words, if that term is 3 years, then the data cannot be altered in any way for the remainder of the term that it has to be kept, say, a further 7 years.
• All persons about whom data is held, or archived, must be made aware of what is contained in that electronic archive and have access to their personal data
• Email and instant messaging must be retained in its entirety and must be managed and treated in the same way as personal data
• All data held must be correct
• All data held must be kept secure with appropriate access restrictions confined to those with legitimate reasons for having access to it.
Which all in all makes data retention an exact science – all organisations have to get this right. Needless to say that if legislation says you have to comply with x then every so often someone is going to want to make sure an organisation has done right by x. Failure to do so in Europe will mean criminal sentences for CEOs and board members, heavy company fines, long drawn-out litigation and businesses going under. It’s like Y2K all over again, but a real threat and on-going forever.
So, the first thing an organisation needs to do is look at its current electronic data/document retention policies and abilities and try to measure the gap between its ability to comply and the compliancy requirement. Fundamentally every organisation needs to consider its data retention and retrieval policies on a global basis. It is not just about whether such and such legislation is required in its own country, nor is it whether or not an organisation wants to do business in this or that country. Legislation around this subject will inevitably become global in nature, so even if storing data is currently not legislation in say, Latvia, it won’t be long before it is.
Most countries have legislation that gives individual access to data, sets up data management practices, guarantees security of data, ensures that individuals can request copies of their own data and gives a right of privacy to individuals - so businesses should start thinking along these lines. Most countries do not have specific retention periods set down by its own limitation rules. However, the statutory limitation periods are relevant because they indicate the length of time documents will be required to bring or defend proceedings however, if, as in many countries, there are rules that certain documents should always be kept in original form and never destroyed then this point becomes irrelevant. Many countries have law on specific vertical markets, or specific company law, or regulatory requirements pertaining to certain company documents that must be retained for fixed periods.
Now, although organisations could approach the problem with a wide sweeping statement that all data must be kept forever, not all data is equal – some is more equal than others. And although by addressing data retention legislation, which suggests that all data is important to all organisations, forever, there is less inherent value to a business in data as it gets older. The idea behind data information life cycle management, therefore, is that data has different values at different moments in time. When an email first arrives in your in-box the information may be urgent or indeed vital for securing a business deal or transaction. After 2 or 3 days the longer the information held in that email may be of little significance and so it’s importance to the recipient becomes less and less. However, that information might have value to other organisations or individuals even if they're no longer useful to the original recipient.
The life cycle of Data begins with a business need for acquiring data. Delivering critical business information into the hands of the people who need to make both strategic and tactical decisions every day is an important trend in today’s market. Active data is referenced on a regular basis during day-to-day business operations. Over time, this data loses its importance and is accessed less often, gradually losing its business value, becoming inactive and eventually ending with its archival or disuse. But even in its archived state, data still has some inherent value to a business. The simple, but vital principle, that all data moves through life-cycle phases, is the starting point for improving data management. By understanding how data is used and how long it must be readily available, companies can develop a strategy, by using policy based management, to map usage patterns to the optimal storage media, thereby minimising the total cost of storing data over its life.
Active Data
Data becomes active as soon as it is captured by an organisation or becomes of business interest to it. This is usually at entry point into the organisation. This data must be accessible easily, swiftly to the business user in order for organisations to run efficient businesses. The same principle applies when data is stored in relational databases, although managing and storing relational data is compounded by complexities inherent in data relationships. Relational databases are a major consumer of storage and are also among the most difficult to manage because they are accessed on a regular basis. Without the ability to manage relational data effectively, relative to its use and storage requirements, runaway database growth will result in increased operational costs, poor performance, and limited availability for the applications that rely on these databases. The ideal solution is to manage data stored in relational databases as part of an overall enterprise data management solution. Data deteriorates at around 10% per month, in other words, for any single record item, 10% will alter within a month. So unless a record is in constant use (this could be customer records for example), its value to the business will deteriorate rapidly and will cease to be active.
Inactive Data
Once data ceases to be active it becomes inactive data. This data is no longer business critical and can be migrated to cheaper storage. Inactive data is old files no longer accessed, files that may be of use in the future but are no longer accessed on a regular basis. Inactive data can be from 1 year to 3 years old. It is still necessary to be able to access and retrieve this type of data for compliance purposes but is likely to have less inherent value to the business.
Archived Data
Archived data is information that is no longer an immediate use to the business. Prior to the mid-nineties, most organisations archived data in Microfilms and tape back-ups. There are now technologies for data archival such as Hierarchical Storage Management, Data Lifecycle Management and Information Lifecycle Management, as well as the supporting infrastructure and media technologies such a NAS, SAN, Serial ATA, etc….. These storage systems can maintain referential integrity and business context. Furthermore backup vaulting can help to manage the process of storing and retrieving data as well as enabling a dramatic reduction in backup and restore times as well as media utilisation as data that is captured in the archive, no longer needs to be backed up regularly.
Comments
aspen
Not applicable
- Mark as Read
- Mark as New
- Bookmark
- Permalink
- Report Inappropriate Content
06-30-2009
09:30 AM
Businesses should be aware when writing any data retention policy that they can be setting themselves up for legal liability. When you set a policy you become responsible for it and deviation from policy will hurt you in lawsuits.
Data retention policies also become double edged swords when a lawsuit hits. Forcing your organization to unintentionally reveal unrelated classified internal data to satisfy the requirements of the discovery phase of a lawsuit.
As a note to any fledgling data retention manager, data can assist and harm you. If you dont create a policy that serves to protect your business, it may cause more damage to the organization than it is worth.
Data retention policies also become double edged swords when a lawsuit hits. Forcing your organization to unintentionally reveal unrelated classified internal data to satisfy the requirements of the discovery phase of a lawsuit.
As a note to any fledgling data retention manager, data can assist and harm you. If you dont create a policy that serves to protect your business, it may cause more damage to the organization than it is worth.
