I’m still fairly new at working with Go. I have the goal of working with, and perhaps even on, several open source projects that use Go as their implementation language. It seems like the number of open source projects using Go is increasing rapidly. It also seems like I need to get acquainted more with Go, and find out how to interact from Go with Oracle Cloud Infrastructure (OCI), as I intend to add features to open source products that leverage OCI services.

Go application for interacting with the OCI ObjectStorage Service

In this article we’ll get a quick introduction to interactions with the OCI Object Storage Service for the purpose of creating and removing a bucket, and an object from a simple Go program that uses the OCI Go SDK. My environment is Ubuntu 20.4 on WSL2 on Windows 10, although that shouldn’t matter much.

The steps in this article:

Install a Go environment.
Create a Go application to connect to OCI using the OCI SDK.
Interact with OCI Object Storage.

Install Go

Using this article: Install Go on Ubuntu — https://buildvirtual.net/how-to-upgrade-go-on-ubuntu/ — I quickly got the current version of Go removed and the latest version of Go installed.

Removing original Go version installed on Ubuntu 20.4

Go get the URL for the latest release: https://go.dev/dl/

Download using wget and untar

Download the Go binary for Linux 64bit and extracting the tar-ball

Then move the extracted directory go to the /usr/local directory:

I tried to run go at this point, but I was too early. First, a small edit of .profile is in order: add the definition of environment variable GOPATH and add $GOPATH to the PATH environment variable.

Once the change is made to .profile and the change has been applied with source ~/, the profile environment is once again ready to run Go with the latest version.

Create a Go application to connect to OCI using the OCI SDK

On the Linux command line, I created a new directory — oci-client — and with “code .” I launched VS Code on this directory.

I can edit code in VS Code and run the code in the context of the Ubuntu environment with the Go runtime set up. Let’s create the simplest of Go applications.

And run it!

No interaction with OCI yet, but a running Go application is a nice start.

Now create file go.mod and add this contents:

Copy code snippet

Copied to Clipboard

Error: Could not Copy

Copied to Clipboard

Error: Could not Copy

<span class="et ua ta hj ub b ch uc ud l ue" data-selectable-paragraph="" id="9e98">module oci-client
go 1.16
require github.com/oracle/oci-go-sdk/v54 v54.0.0</span>

Then run: “go mod tidy” (see docs for help): go mod tidy ensures that the go.mod file matches the source code in the module. It adds any missing module requirements necessary to build the current module's packages and dependencies, and it removes requirements on modules that don't provide any relevant packages. It also adds any missing entries to go.sum and removes unnecessary entries. "

Define constants in the application for the tenancy OCID, the user OCID, the region and the fingerprint (the usual entries in the oci config file):

Import these packages:

Extend the application to the following listing — with functions initializeConfigurationProvider (to contact OCI) and listDomains (to interact with the identity service APIs

Finally create a file — for example called ppk (putty private key) — that contains the private key for an OCI account in PEM format.

With this in place, I can run the application.

The first action is to export an environment variable with the contents of the file ppk as its value:

Copy code snippet

Copied to Clipboard

Error: Could not Copy

Copied to Clipboard

Error: Could not Copy

<span class="et ua ta hj ub b ch uc ud l ue" data-selectable-paragraph="" id="c919">export pem=$(cat ./ppk)</span>

This makes the private key accessible to the Go application as environment variable, using os.Getenv(“pem”).

Export the environment variable pem with the contents from file ppk

And there is contact between my locally running Go application and the OCI — as the output shows.

Interact from Go Application with OCI Object Storage

Time now to introduce the Object Storage APIs in the SDK (see docs here). These will allow us to create buckets and objects in those buckets from the Go application. This is a capability I’ll need when I implement a custom Dapr state component leveraging OCI Object Storage — as is my goal.

Let’s run the Go application I have crafted:

Run the Go application discussed in this article

The result of running this code is the creation of a Bucket:

Bucket GoBucket is created on OCI Object Storage

and a new file with the specified contents in this bucket:

Object FreshObject is created in the bucket GoBucket

Inspect the contents by downloading and view the object to my local file system:

Download the object through the OCI Console to verify its contents 👆

So my Go application works with OCI Object Storage.

The main function in the application orchestrates the moves:

The main function that orchestrates the actual work is performed in this application 👆

The configurationProvider contains the OCI connection details (such as OCIDs for tenancy and user, the fingerprint and private key). This provider is used to create the objectStorageClient — which is used for all subsequent interactions with the OCI ObjectStorage service.

The steps subsequently taken:

Determine the namespace for the tenancy.
Ensure that a bucket with the specified name exists — check if it exists already and if it does not, then create it.
Put an object with the specified name and contents in the bucket (overwrite the object that may already be there under that name).
Defer the statement that will remove the object again.
Retrieve the object that was just created and write its contents to the output.
Sleep for one minute and afterwards execute all deferred statements (which means that the object is removed).

This code is not robust, it does hardly any error checking and handling. It most certainly cannot be used in anything like a production environment. However, it does illustrate how we interact with the ObjectStorage service through the Go SDK, and under the right circumstances it certainly does the job of creating the bucket and the object plus reading the object and removing it.

Let’s look at the code in a little more detail. This code for initializing the OCI Configuration Provider is generic across all Go applications that interact with whatever OCI services. It uses common OCI configuration details — commonly found in the config file in directory .oci when using the OCI CLI — to create signed HTTP requests to the OCI REST APIs

The code that checks if a bucket already exists — by getting it and checking on status code 404 (not found) and creating the bucket if it was not found — is shown below. Note that the getBucketRequest struct expects pointers to strings for namespacename and bucketname (see docs), hence the &namespace and &name to produces the addresses for these string values (one of the intricacies of Go I am still getting used to)

With the Bucket established, creating the object is done as follows:

This is quite straightforward code — except for the way in which the string that contains the object’s content is turned into a reader object (at least to me that looks a bit special).

To retrieve an object from the OCI Object Storage service is basically the reverse operation from putObject:

This function returns the contents of the object as a string — assuming that it is at all meaningful to turn this object into a string. Note that the response object returns pointers to ContentLength, ContentType and Content. The asterisk (*response.Content:Length) is used to retrieve the actual value (at the pointer’s location).

Finally to delete the object — the simplest of the operations:

The full code for my OCI ObjectStorage sample client is shown below

Copy code snippet

Copied to Clipboard

Error: Could not Copy

Copied to Clipboard

Error: Could not Copy

package main

import (
  "context"
  "fmt"
  "io"
  "io/ioutil"
  "log"
  "net/http"
  "os"
  "strings"
  "time"

  "github.com/oracle/oci-go-sdk/v54/common"
  "github.com/oracle/oci-go-sdk/v54/objectstorage"
)

const tenancyOCID string = "ocid1.tenancy.oc1..aokq"
const userOCID string = "ocid1.user.oc1..aaaaaaa"
const region string = "us-ashburn-1"
const fingerprint string = "02:91:6c:49:d8:04:56"
const compartmentOCID = "ocid1.compartment.oc1..aaaaazrq"

func initializeConfigurationProvider() common.ConfigurationProvider {
  privateKey := os.Getenv("pem") // set content of PEM in environment variable pem using export pem=$(cat ./ppk)  with ppk a text file that contains the PEM private key00
  configurationProvider := common.NewRawConfigurationProvider(tenancyOCID, userOCID, region, fingerprint, string(privateKey), nil)
  return configurationProvider
}

// fatalIfError is equivalent to Println() followed by a call to os.Exit(1) if error is not nil
func fatalIfError(err error) {
  if err != nil {
    log.Fatalln(err.Error())
  }
}

func getResponseStatusCode(response *http.Response) int {
  return response.StatusCode

}

// bucketname needs to be unique within compartment. there is no concept of "child" buckets.
func ensureBucketExists(ctx context.Context, client objectstorage.ObjectStorageClient, namespace, name string, compartmentOCID string) {
  req := objectstorage.GetBucketRequest{
    NamespaceName: &namespace,
    BucketName:    &name,
  }
  // verify if bucket exists
  response, err := client.GetBucket(context.Background(), req)
  if err != nil {
    fatalIfError(err)
    if 404 == response.RawResponse.StatusCode {
           createBucket(ctx, client, namespace, name, compartmentOCID)
    }
  }
}

// bucketname needs to be unique within compartment. there is no concept of "child" buckets.
func createBucket(ctx context.Context, client objectstorage.ObjectStorageClient, namespace string, name string, compartmentOCID string) {
  request := objectstorage.CreateBucketRequest{
    NamespaceName: &namespace,
  }
  request.CompartmentId = &compartmentOCID
  request.Name = &name
  request.Metadata = make(map[string]string)
  request.PublicAccessType = objectstorage.CreateBucketDetailsPublicAccessTypeNopublicaccess
  _, err := client.CreateBucket(ctx, request)
  fatalIfError(err)
  fmt.Println("Created bucket ", name)
}

func getNamespace(ctx context.Context, client objectstorage.ObjectStorageClient) string {
  request := objectstorage.GetNamespaceRequest{}
  r, err := client.GetNamespace(ctx, request)
  fatalIfError(err)
  return *r.Value
}

func putObject(ctx context.Context, c objectstorage.ObjectStorageClient, namespace, bucketname, objectname string, objectContent, metadata map[string]string) error {
  request := objectstorage.PutObjectRequest{
    NamespaceName: &namespace,
    BucketName:    &bucketname,
    ObjectName:    &objectname,
    ContentLength: &int64(len(objectContent)),
    PutObjectBody: ioutil.NopCloser(strings.NewReader(objectContent)),
    OpcMeta:       metadata,
  }
  _, err := c.PutObject(ctx, request)
  fmt.Println("Put object ", objectname, " in bucket ", bucketname)
  return err
}

func getObject(ctx context.Context, c objectstorage.ObjectStorageClient, namespace string, bucketname string, objectname string) (string, error) {
  fmt.Println("get object ", objectname)
  request := objectstorage.GetObjectRequest{
    NamespaceName: &namespace,
    BucketName:    &bucketname,
    ObjectName:    &objectname,
  }
  response, err := c.GetObject(ctx, request)
  // fmt.Println("get object, status code ", response.RawResponse.StatusCode)
  // fmt.Println("content length ", *response.ContentLength)
  // fmt.Println("content type ", *response.ContentType)
  buf := new(strings.Builder)
  _, err = io.Copy(buf, response.Content)
  return buf.String(), err
}

func deleteObject(ctx context.Context, c objectstorage.ObjectStorageClient, namespace, bucketname, objectname string) (err error) {
  request := objectstorage.DeleteObjectRequest{
    NamespaceName: &namespace,
    BucketName:    &bucketname,
    ObjectName:    &objectname,
  }
  _, err = c.DeleteObject(ctx, request)
  fatalIfError(err)
  fmt.Println("Deleted object ", objectname)
  return
}

func main() {
  configurationProvider := initializeConfigurationProvider()
  objectStorageClient, clerr := objectstorage.NewObjectStorageClientWithConfigurationProvider(configurationProvider)
  fatalIfError(clerr)
  ctx := context.Background()
  namespace := getNamespace(ctx, objectStorageClient)

  bucketName := "GoBucket"
  ensureBucketExists(ctx, objectStorageClient, namespace, bucketName, compartmentOCID)
  objectName := "FreshObject"
  objectContent := "My Content in plain text"
  err := putObject(ctx, objectStorageClient, namespace, bucketName, objectName, objectContent, nil)
  fatalIfError(err)

  // remove the object after 60 seconds - by first deferring the removal and then waiting for 8 seconds
  defer deleteObject(ctx, objectStorageClient, namespace, bucketName, objectName)

  fmt.Println("go get object ", objectName)
  contents, error := getObject(ctx, objectStorageClient, namespace, bucketName, objectName)
  fatalIfError(error)
  fmt.Println("Object contents: ", contents)

  fmt.Println("Go Sleep")
  time.Sleep(60 * time.Second)
  // when sleep is over, deferred statements are executed prior to exiting the application
}