Clustered file system tiering to the cloud

The amount of data created each year is growing at a pace that was never seen before. More data was generated in the last two years than in the entire human history before that. Naturally, the cost of storing and managing the data has also skyrocketed and shows an upward trend resulting in a drive to optimize storage costs.

One way to lower costs is to move stale data to lower-cost storage, such as the public cloud.  If done manually this can be a timely and ongoing labor-intensive process to manage.  Instead, the Veritas file system (VxFS) includes file-level cloud storage tiering that uses policies to automate tiering of data to optimize storage costs and eliminate administrative overhead. It provides a hybrid storage environment that seamlessly integrates local on-premise block storage with cloud object storage, along with a mechanism to migrate data between them. This results in freeing up the expensive DAS and SAN storage for active mission-critical data or high-performance applications, while less-frequently used data resides in the cloud tier.Blog_Cloud_Tiering_LI_2-02.png

How VxFS tiering works?

VxFS clustered file system tiering provides a method to assign different categories of data to various types of storage media, including the cloud, with the objective of reducing the total cost of storage. VxFS’s tiered storage architecture places data in a hierarchy with tiers determined by the performance and cost of media and the data placement amongst the tiers are ranked by how often users access it.

It works by storing the file’s metadata on-premise and the file's data is stored as fixed-size objects in the cloud. The nomenclature of the object has been done in such a way that it reflects some metadata about the file, which also allows the creation of metadata for already existing cloud objects.

How to Tier Data with VxFS file-level tiering

The movement of data is policy-based. Users can assign a policy to a tier based on factors such as size, age, modification time, and access time. After a policy is assigned to a tier, all the files within the tier satisfying the policy are automatically relocated. Relocation to different tiers of cloud storage and to different cloud vendors are possible such as Amazon S3 and S3 compatible, Amazon Glacier, Microsoft Azure, and Google cloud.

Amazon Glacier is a great storage choice when storage cost is paramount, data is rarely accessed, and retrieval latency of several hours is acceptable. Data can be stored cost-effectively for months, years, or even decades. If the application requires fast or frequent access to data other cloud storage such as Amazon S3, Google Cloud, or Azure can be used.

Disaster recovery and data resiliency

If the system crashes while the data is being transferred, VxFS guarantees that the data will be in a consistent state when the application comes back online. Based on the data transfer progress at the time of the crash, each file will be present on either local storage or cloud storage.Figure2.pngVxFS stores mapping information between cloud objects and files, on primary storage, so the availability of cloud data is limited by the accessibility of primary storage. To address this, block-level replication is used which supports both synchronous and asynchronous replication methods, which ensures that data will always be available. If a disaster occurs at the source location, the copy of the replicated data at the remote location is used to restart the application. Since cloud providers provide data reliability, a copy of the data stored in objects in the cloud is not required in VxFS.

Conclusion

The big data growth we’ve been witnessing is only natural, we constantly generate data. Moving infrequently used data to the cloud can improve a business’s agility and shorten payback times. To easily leverage this benefit VxFS file-level cloud tiering provides a single platform that can intelligently migrate data to the cloud.  InfoScale also provides different features on files stored in the cloud such as volume replication, smart I/O.

For more information on file system tiering check out: https://www.veritas.com/content/support/en_US/doc/130803809-141542355-0/v125135006-141542355