X

The latest cloud infrastructure announcements, technical solutions, and enterprise cloud insights.

Deploy Lustre File System on Oracle Cloud Infrastructure

Pinkesh Valdria
Principal Solutions Architect

Lustre is an open source, parallel, distributed file system, generally used for high-performance computing (HPC) clusters and environments. The name Lustre is a portmanteau of Linux and cluster.

Using Lustre, you can now build an HPC file server on Oracle Cloud Infrastructure’s bare metal Compute and network-attached block storage or NVMe SSDs locally attached to Compute nodes. To make it easier to deploy Lustre on Oracle Cloud Infrastructure, we've released a new Terraform template.

It's easy to scale a Lustre cluster for higher throughput, higher storage capacity, or both for the file system, and it costs a few cents per gigabyte per month (compute and storage combined).

Deployment

The Terraform deployment template provisions Oracle Cloud Infrastructure resources (compute, storage, virtual cloud networks, subnets, and so on) and Lustre file system software, which includes a Management Server (MGS), Metadata Server (MDS), Object Storage Server (OSS), and Lustre client nodes.

The scalable Lustre architecture has the following components:

  • The Management Server (MGS) is a global resource that can support multiple file systems. The MGS stores configuration information for one or more Lustre file systems and provides this information to other Lustre hosts.
  • Metadata Servers (MDS) provide the index, or namespace, for a Lustre file system. The metadata content is stored on volumes called Metadata Targets (MDTs). A Lustre file system’s directory structure and file names, permissions, extended attributes, and file layouts are recorded to MDTs. Each Lustre file system must have a minimum of one MDT.
  • Object Storage Servers (OSS) in a Lustre file system provide the bulk data storage for all file content. Each OSS provides access to a set of storage volumes referred to as Object Storage Targets (OSTs). Each OST contains a number of binary objects that represent the data for files in Lustre. Files in Lustre are composed of one or more OST objects, in addition to the metadata inode stored on the MDS.
  • Lustre clients are compute instances that access the Lustre file system.

You can customize the Terraform template based on your needs. For example, you can customize the following resources:

  • Number of Metadata Servers (MDS) and their compute shapes
  • Number of Metadata Targets (MDT) and their size (in GB)
  • Number of Object Storage Servers (OSS) and their compute shapes
  • Number of Object Storage Targets (OST) and their size (in GB)
  • Whether the Management Server (MGS) is deployed on the first MDS node

The following diagram shows the Lustre file system architecture on Oracle Cloud Infrastructure.

Diagram that shows the Lustre file system architecture on Oracle Cloud Infrastructure.

The file system can be created with bare metal or virtual machines (VM) Standard compute shapes. However, we recommend using bare metal compute shapes because they have two physical NICs each with 25 Gbps network speed. You can you use one NIC for all traffic to block storage and use the other NIC for incoming data to the OSS and MDS nodes from client nodes. For more information about compute shapes, see the Compute page.

The bare metal compute instances are connected in clusters to a nonoversubscribed 25-gigabit network infrastructure, providing extremely low latency and very high throughput, which is a key requirement for a high-performance file system. In fact, Oracle Cloud Infrastructure is the only cloud with a network throughput performance SLA.

For storage, you can either use our block volume storage or local NVMe SSDs that are directly attached to the compute nodes. Our block storage is backed by NVMe, and block storage performance is backed by an SLA. Each volume size of 1 TB and above provides 480 MB/s, by default at no additional charge. To use our locally attached NVMe SSDs, select DenseIO bare metal or VM compute shapes. For more information, see the Block Volumes page and the Oracle Block Volume performance metrics and local NVMe storage metrics blog posts.

The Terraform template will soon be made available on GitHub.

Try It Yourself

Every use case is different. One way to know if Oracle Cloud Infrastructure is right for you is to try it. You can select either the Oracle Cloud Free Tier or a 30-day free trial that includes US$300 in credit to get you started with a range of services, including compute, storage, and network. Also, visit us at booth 609 at the SuperComputing 2019 (SC19) conference on November 17–22 in Denver, Colorado.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha