OCI Object Storage validated with IBM Spectrum Scale as low-cost storage tiering layer

March 28, 2022 | 4 minute read
Pinkesh Valdria
Principal Solutions Architect
Text Size 100%:

OCI REST APIs for Object Storage

Oracle Cloud Infrastructure (OCI) offers multiple REST APIs to access Object Storage service. One of them is designed to provide Amazon S3 compatibility. Using the OCI Amazon S3 Compatibility API, you can continue to use your existing Amazon S3 tools, such as SDK clients, and make minimal changes to your applications to work with OCI Object Storage.

The OCI Amazon S3 Compatibility API and OCI native Object Storage API datasets are congruent. If data is written to the Object Storage using the Amazon S3 Compatibility API, the data can be read back using the native Object Storage API.

AFM to cloud object storage

IBM Spectrum Scale has an AFM to cloud object storage feature that enables placement of files or objects in an IBM Spectrum Scale cluster to a cloud object storage. AFM to cloud object storage supports Amazon S3 and IBM cloud object storage. Because OCI Object Storage also includes Amazon S3 Compatibility API, you can also use the Spectrum Scale AFM to cloud object storage feature to integrate with OCI Object Storage.

Depending on the modes of AFM to cloud object storage fileset configurations, you can download, work on, and upload objects required for applications, such as AI, ML, and big data analytics, to a cloud object storage. You also synchronize the objects created by applications to the objects on a cloud object storage asynchronously. An AFM to cloud object storage fileset can cache only metadata or both metadata and data.

To use the AFM to cloud object storage feature of Spectrum Scale on OCI, use IBM Spectrum Scale release 5.1.2 or higher, because this release includes changes required to integrate with OCI Object Storage S3 Compatibility API.

You can also use this feature to reduce and control cost of Spectrum Scale file system by freeing the IBM Spectrum Scale file system storage capacity by moving less useful data to the cloud storage.

Use cases

This feature enables following use cases:

  • Transparent cloud tiering: POSIX access to endless amount of OCI Object Storage data, such as high-performance computing (HPC) workloads

  • Caching: Cache object storage data in Spectrum Scale for faster computation and synchronize back to object storage, such as AI and ML training data sets and models

  • Use Object Storage as the long term highly redundant, low-cost storage for your AI, ML, HPC, and File Server data and use Spectrum Scale file system on OCI to run IO heavy workloads.

  • Multiple remote sites, satellite work, and one primary data copy in Object Storage and cached in multiple remote Spectrum Scale clusters, such as media entertainment workloads (audio and video editing)

Integration with OCI Object Storage 

The following steps integrate Object Storage to Spectrum Scale. This process assumes that you already have Spectrum Scale File System installed and configured for AFM gateway nodes.

  1. Create a bucket on OCI object storage and set S3 compatibility API compartment to what you plan to use: afm-ocios-hpc.

  2. Set up an access key and a secret key to access a bucket on a cloud object storage.

    [root@dr-compute-1 ~]# mmafmcoskeys afm-ocios-hpc:us-phoenix-1@hpc.compat.objectstorage.us-phoenix-1.oraclecloud.com set e4xxxxxxxxxxea4d853c6b8123 actualsecretkey
  3. Validate that keys were configured correctly.

    mmafmcoskeys afm-ocios-hpc:us-phoenix-1@hpc.compat.objectstorage.us-phoenix-1.oraclecloud.com get e4xxxxxxxxxxea4d853c6b8123:actualsecretkey
  4. Do a bucket lookup to ensure that the setup works.

    /usr/lpp/mmfs/bin/mmafmtransfer -t -a actualaccesskey -s actualsecretkey -b afm-ocios-hpc -r us-phoenix-1 -e https://hpc.compat.objectstorage.us-phoenix-1.oraclecloud.com:443 blookup
    echo $?
  5. Map the AFM fileset to the bucket and check the state.

    mmafmcosconfig drfs1 afms3 --endpoint https://us-phoenix-1@hpc.compat.objectstorage.us-phoenix-1.oraclecloud.com --uid 0 --gid 0 --bucket afm-ocios-hpc --mode iw --object-fs --cleanup --debug
    echo $?

    Initially, it might show “Inactive.” Generate some data and check again.

    mmafmctl drfs1 getstate
    Fileset Name Fileset Target Cache State Gateway Node Queue Length Queue numExec
    ------------ -------------- ------------- ------------ ------------ -------------
    afms3 https://us-phoenix-1@hpc.compat.objectstorage.us-phoenix-1.oraclecloud.com:443/afm-ocios-hpc Inactive
  6. Generate data to save on OCI Object Storage.

    /usr/lpp/mmfs/samples/perf/gpfsperf create seq /gpfs/drfs1/afms3/newfile40 -n 100M -r 1M
  7. Check the generated files on the file system.

    ls -l /gpfs/drfs1/afms3/
    drwxr-sr-x 3 root root 8192 Dec 1 02:24 prefix1
    -rw-r--r-- 1 root root 104857600 Nov 30 15:10 newfile40
    -rw-r--r-- 1 root root 104857600 Nov 30 15:24 newfile41
  8. Check the generated files on OCI Object Storage.

    A screenshot of the bucket information for the create bucket.

For more details, reach out to the OCI HPC Storage Team.

Pinkesh Valdria

Principal Solutions Architect


Previous Post

Private endpoints for Oracle services

Troy Levin | 8 min read

Next Post

Announcing new deployment strategies for OCI DevOps Service

Saurabh Shah | 3 min read