X

Bare Metal Cloud Object Storage Overview

Hello there! My name is Rachna Thusoo. I am a Product Manager on the Oracle Bare Metal Cloud Service team. In the recent years, Object Storage has emerged to become the ubiquitous storage of choice to cost effectively and durably store large amounts of data in the cloud. When we announced the general availability of the Bare Metal Cloud Services, BMC Object Storage was amongst the first few services that we launched with. Today I’ll walk you through an overview of the service and provide information on the various interfaces you can use to interact with the Object Storage.

The Bare Metal Cloud Object Storage is a high performance, highly durable, available, and scalable storage platform. It’s been designed from the ground up, keeping in view the principle that proximity of compute and storage on a non-blocking, shared, network fabric is fundamental to driving superior storage performance. It’s a regional service. You can access Bare Metal Cloud Storage from anywhere as long as you have access to an Internet connection.

Bare Metal Cloud Object Storage Service is comprised of the following components

  • Objects: All data, regardless of content type, is stored as an object in the Object Storage Service.
  • Buckets: A bucket is a container that stores objects. You can store objects in one or multiple buckets under a single tenancy.
  • Namespace: A namespace is the logical construct that lets you own your personal bucket namespace. Since bucket names are not global you have more freedom and flexibility with how you assign names to buckets. Each Bare Metal Cloud tenant is associated with one default, system assigned namespace.
  • Compartments:  A compartment is a collection of related resources that can be accessed only by those groups that have been given permission by an administrator to access those resources. Compartment is not necessarily a concept that is exclusively associated with the Object Storage service, but it's pertinent because all buckets exist in a compartment.

Here is how the Object Storage components fit together

Pic1.jpg

In the example above, if a user has been granted access to Compartment A, they will only be able to access Bucket1/Bucket2 and objects that are stored in these buckets.

Here are the steps you need to follow to start using the Bare Metal Cloud Object Storage

Step #1 Create a Bare Metal Service account. You can do so by directly placing an order at https://shop.oracle.com or if you are used to working directly with your Oracle sales rep, reach out to them and they will be able to help you.  Once you receive your Bare Metal Cloud credentials, via a ‘Welcome’ email, you can log into the Bare Metal Service console and start using the service.

Pic2.jpg

 

Step # 2: Create a compartment

Each tenancy is associated with a default compartment which is what will be chosen by default, if you do not create a new compartment. However it’s highly recommended that you give some thought to your choice of the compartment where you want to create your first bucket. Once the choice is made, you will not be able to migrate your buckets to another compartment.

To create a compartment, click on Identity -> Compartments and then follow the 'Create Compartment' workflow.

Pic3.jpg

Pic4.jpg

 

You can also optionally choose to create additional users and enable them to access the compartment that you intend to create your bucket in.

To create a new user, go to the BMC service console, click on Identity -> Users and then follow the “Create User” workflow.  Once created, add this user to a group of your choice and assign the appropriate IAM policies. Assigning IAM policies to a group enables all users in that group to operate on buckets created in a specific compartment.

Example of an IAM policy:

ALLOW group ‘example_group’ to manage all-resources ON compartment ‘Dev_Test’

Step # 3: Create a Bucket

From the Bare Metal Service Console, navigate to Storage -> Object Storage and follow the 'Create Bucket' work flow to create your first bucket. Be sure to follow the correct bucket naming convention (Bucket Naming Convention)

Pic5.jpg

Pic6.jpg

Step # 4: Upload an object

Select the bucket you just created and from the bucket management interface, follow the 'Upload Object' work flow to start uploading your objects in the buckets.

Pic7.jpg

To upload objects, you can either browse your local folder for objects to be uploaded, by clicking on the 'Upload Object' button, or you can simply drag and drop files into the upload object interface. The Bare Metal Cloud Object Storage supports a maximum object size of 50 GB per object, however that limit is slightly smaller (<= 5GB) when you are uploading objects from the UI.

Pic8.jpg

By carefully choosing the object name to include delimiters, you can simulate a virtual folder hierarchy. For example, objects with the name

2017/invoices/March/Invoice1.doc and 2017/invoices/March/Invoice2.doc would emulate a traditonal storage file system folder hierarchy.

Pic9.jpg

Congratulations! You just successfully created your first bucket and uploaded your first object to the Bare Metal Cloud Object Storage.  That was all fairly simple, right?

Object Storage Durability, Availability and Security

Oracle Bare Metal Cloud Object Storage Service is highly secure. It is tightly integrated with the Oracle Bare Metal Cloud Identity and Access Management Service. By default, only authenticated users that have explicitly been granted access to specific resources can access data stored in the Object Storage Service. Data is uploaded and downloaded from the Object Storage Service over SSL endpoints using the HTTPS protocol. For an additional layer of security, you can encrypt objects, prior to sending them to the Object Storage Service. That gives you total control over not only your data, but also the encryption keys that are used to encrypt the data.

The Object Storage Service was also designed to be highly durable. Data is stored redundantly across multiple storage servers across multiple Availability Domains. Data integrity is actively monitored using checksums and corrupt data is detected and auto-repaired. Any loss in data redundancy is automatically detected and self-healed, without any customer impact.

Bare Metal Cloud Object Storage Interface

Though using the Bare Metal Cloud UI is the quickest and easiest way to get up and running on the Bare Metal Cloud Object Storage, to realize the power of using Object Storage at scale, you will want to evaluate the programmatic interfaces as a means of accessing the BMC Object Storage.

REST API

You can make your application, running on-premise or in the cloud interact directly with the Object Storage by integrating with the Bare Metal Cloud RESTful API.  Bare Metal Cloud Object Storage supports two types of APIs

For general purpose use, we recommend that you use the native Bare Metal Cloud Object Storage API. The Swift API primarily exists to provide support for Oracle RMAN backups.

The usual Bare Metal Cloud user authentication mechanisms will work seamlessly with the native API.  However to use the Swift API, you will need to generate a Swift specific password. You can do so from the Bare Metal Cloud Service console by clicking on Identity and then choosing a user that you want to create a password for.  You can get more details on the native BMC Object Storage API interface here

Java SDK

In addition to the BMC native API, a Java SDK exists that helps simplify interactions with the Object Storage. Amongst other things, the Java SDK enables you to delegate the complexity of forming and signing an Object Storage request. In the near future we will augment the SDK portfolio to support other languages, but for now Java is the only supported language.  You can download the Java SDK here

Command Line Interface (CLI)

The command line interface enables you to automate your storage workflows via scripting. If using the Object Storage via the API interface is not practical and you are not ready yet for a full API/SDK integration, the CLI is a happy middle path to use the Object Storage. You can download the CLI bits here. The CLI is fairly easy to use. The general syntax of the CLI follows this pattern:

> bmcs os [OPTIONS] COMMAND [ARGS]...

The supported commands are bucket, ns and object

To get help on any command type --help e.g.   " bmcs os bucket --help "

Example: Create a bucket

     CLI command
>
bmcs os bucket  create -ns acme -c ocidv1:compartment:oc1:phx:1461107489276:aaaaaaaahu7a7bttjpjxla6n3g677wdcs --name bucketname

     Response

"data": {

    "compartment-id": "ocidv1:compartment:oc1:phx:1461107489276:aaaaaaaahu7a7bttjpjxla6n3g677wdcsu",

    "created-by": "ocidv1:user:oc1:phx:1461016901591:aaaaaaaaxhcqkaofal5zdtfhvl4m6bp5fy",

    "etag": "8b03c03b-9712-40b2-b2c8-5e3ea93e79ae",

    "metadata": {},

    "name": "bucketname",

    "namespace": "acme",

    "time-created": "2016-12-14T23:25:03.693000+00:00"

  },

  "etag": "8b03c03b-9712-40b2-b2c8-5e3ea93e79ae"

}

Example: Upload an object

> bmcs object put -ns acme -bn bucketname --name image01.jpg -file image01.jpg

Response:

{

  "etag": "4AB9BB832396561FE053C20610ACBA2E",

  "last-modified": "Tue, 14 Dec 2016 23:26:52 GMT",

  "opc-content-md5": "1B2M2Y8AsgTpgAmY7PhCfg=="

}

HDFS Connector

You can use the Bare Metal Cloud Service HDFS connector to enable your big data workloads to interact directly with the Object Storage and to use it as the primary file system. The HDFS connector works with analytics engines like Hadoop and Spark. You can download the jar file to use with your Hadoop installation here.

To summarize, Bare Metal Cloud Object Storage is an ideal storage platform to cost effectively and durably store petabyte and exabytes of data.  It’s a foundational service for any cloud native workload and an efficient means of storing burgeoning data generated by traditional workloads.  The availability of a wide variety of management interfaces will enable you to choose the level of integration that you are comfortable with. The Object Storage service helps you delegate your concern around data durability and availability to the Bare Metal Cloud platform. Once ingested, you can be sure that the data will be available, whenever you need to access it.

As I am writing this blog, we are feverishly working behind the scenes to enhance the Object Storage offering. I would highly recommend bookmarking this page, because we will continuously be updating this blog space as we launch new features (Teaser : There are a lot of juicy new features coming soon). In my next blog I’ll cover the various use cases you might want to consider using Object Storage for. More specifically I’ll provide a detailed account of how you can use the Bare Metal Cloud Storage with Big Data Analytics engines. If you have other suggestions, around features and capabilities that you would like me to cover, please comment below. I'll be happy to provide coverage in my future blog posts.

Rachna Thusoo
Director, Product Management
Oracle Bare Metal Cloud Team

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha
Oracle

Integrated Cloud Applications & Platform Services