Amazon FSx for Lustre Intelligent Tiering
The newest storage class from Amazon FSx for Lustre offers the only fully elastic Lustre flannel storage at the lowest possible price.
AWS is introducing today the general release of the Amazon FSx for Lustre Intelligent Tiering, a new storage class that offers the lowest cost Lustre file storage in the cloud, the only fully elastic Lustre file storage, and nearly infinite scalability. The cheapest high-performance file storage in the cloud is provided by FSx for Lustre Intelligent-Tiering, which starts at less than $0.005 per GB-month and can save up to 96% on storage expenses for rarely visited data when compared to other managed Lustre choices. Because your file system will expand and contract as you add or remove data, elasticity eliminates the need for upfront storage capacity provisioning and allows you to just pay for the volume of data you keep.
By allocating cold data to the appropriate lower-cost storage tier according to access patterns, FSx for Lustre Intelligent-Tiering automatically optimizes costs. It also comes with an optional SSD read cache to boost performance for those workloads that are most sensitive to latency. Regardless of whether you are working with massive petabyte-scale datasets or gigabytes of experimental data for your most demanding HPC and artificial intelligence/machine learning (AI/ML) applications, Intelligent Tiering provides great performance.
Intelligent Tiering provides up to 34% better price performance than on-premise HDD file systems, and it allows you to modify the performance of your file system regardless of storage. Workloads that combine hot and cold data that are HDD-based or mixed HDD/SSD are best suited for the Intelligent-Tiering storage class. With FSx for Lustre Intelligent-Tiering, you may move and execute these workloads without modifying any applications, doing away with the need to plan and manage storage capacity and only paying for the resources you use.
Before this launch, clients used the FSx for Lustre SSD storage class to speed up HPC and machine learning applications that require consistent low-latency access to all data and all-SSD performance. All-SSD storage is not necessary for cooler data sections, though, as many applications contain both hot and cold data. In AI/ML workloads, FSx for Lustre is being utilized more and more to boost GPU utilization. Being one of the solutions for these workloads is now much more economical.
FSx for Lustre Intelligent-Tiering
Frequent Access, Infrequent Access, and Archive are the three storage tiers that your data transfers between without your intervention, saving you money automatically and without requiring any prior payments or obligations. The way tiering operates is as follows:
Frequent Access: Information kept in this tier has been accessed during the previous 30 days.
Infrequent Access: This tier saves 44% on costs compared to Frequent Access and stores data that hasn’t been accessed in 30 to 90 days.
Archive: At a 65 percent cost decrease over Infrequent Access, data that hasn’t been accessed in 90 days or more is kept in this tier.
No matter the storage tier, unlike traditional on-premises deployments, which are typically limited to a single physical location, your data is stored across various AWS Availability Zones for redundancy and availability. Furthermore, your data can be recovered in milliseconds.
Creating a file system
To construct a file system, you can use the AWS Management Console, AWS CLI, API, or AWS CloudFormation. “Persistent, Intelligent Tiering” is the deployment and storage class that you choose while constructing a file system. You also select the Metadata IOPS and the required throughput capacity. Following the designated throughput capacity, the SSD read cache is automatically configured.
Throughput capacity, metadata IOPS, security groups, and SSD read cache are among the options that can be changed after construction. By paying only for the resources consumed, users can later enhance throughput capacity in accordance with workload requirements.
For redundancy and availability, data is spread across several AWS Availability Zones in any storage tier.
Important points
FSx for Lustre Intelligent Tiering can provide a total throughput of many terabytes per second.The throughput of FSx for Lustre per client using GPU Direct Storage (GDS) and Elastic Fabric Adapter (EFA) support is up to 12 times (up to 1200 Gbps) higher than that of earlier FSx for Lustre systems.
The maximum IOPS it can provide for writes and cached reads is tens of millions. SSD read cache has submillisecond time-to-first-byte latencies, but all other data has tens of milliseconds.
Availability
Let’s remember the following two points:
The new FSx for Lustre storage class is available in the AWS regions of Asia Pacific (Hong Kong, Mumbai, Seoul, Singapore, Sydney, Tokyo), Europe (Frankfurt, Ireland, London, Stockholm), Canada (Central), and the US East (N. Virginia, Ohio), as well as the US West (N. California, Oregon).
You pay in gigabytes per month for the data and metadata that you keep on your file system. Data that is not in the SSD read cache is read or written, and you are charged on an operation basis. Data and metadata SSD read cache size (GB/month), metadata IOPS (IOPS/month), and total throughput capacity (MBps/month) are charged on your file system.
You can also read AWS Introduces AWS MCP Servers for Serverless, ECS, & EKS
Thank you for your Interest in Cloud Computing. Please Reply