In this blog post I’ll show how to use rclone on Oracle Linux with free object storage services included in Oracle Cloud Free Tier. Free tier includes 20GiB of object storage.
Rclone is a command line program to sync files and directories to and from various cloud-based storage services. Oracle Cloud Object Storage is Amazon S3 compatible, so I’ll use Rclone’s S3 capabilities to move data between my local Oracle Linux system and object storage.
One way to configure Rclone is to run rclone config and step through a series of questions, adding Oracle Cloud Object Storage as an S3 compatible provider. Instead, I’m going to use Oracle Cloud Infrastructure’s Cloud Shell to gather the relevant data and construct what’s ultimately a small configuration file.
The high level steps are:
One of the benefits of Cloud Shell is that it includes pre-configured OCI Client tools so you can begin using the command line interface without any configuration steps.
Starting in Cloud Shell, set up environment variables to make running subsequent commands easier. The following stores your region and tenancy OCID, and storage namespace in environment variables. I'm using both JMESPath and jq to parse JSON for illustration purposes.
export R=$(curl -s http://169.254.169.254/opc/v1/instance/ | jq -r '.region') export C=$(oci os ns get-metadata --query 'data."default-s3-compartment-id"' --raw-output) export N=$(oci os ns get | jq -r '.data')
To create a storage bucket:
oci os bucket create --name mybucket --compartment-id $C
The Amazon S3 Compatibility API relies on a signing key called a Customer Secret Key.
export U=$(oci os bucket list --compartment-id=$C --query 'data [?"name"==`mybucket`] | ."created-by"' --raw-output) oci iam customer-secret-key create --display-name storagekey --user-id $U export K=$(oci iam customer-secret-key list --user-id $U | jq -r '[.data | select (."display-name"=="storagekey")]."id"')
In the response from oci iam customer-secret-key, id corresponds to the access key and key represents the secret key. Make a note of the key immediately because it will not be shown to you again!
Finally, gather up the relevant values for the Rclone configuration. Remember to copy the secret key and save it somewhere.
Run the following to collect and display the information you need.
echo "ACCESS KEY: $K"; echo "SECRET KEY: check your notes"; echo "NAMESPACE: $N"; echo "REGION: $R"
Over to the local system on which Rclone will be used to move files to- and from object storage.
To install Rclone:
$ sudo yum install -y oracle-epel-release-el7 && sudo yum install -y rclone
In your home directory, create a file, .rclone.conf using the contents below, replacing the values you gathered earlier:
[myobjectstorage] type = s3 provider = Other env_auth = false access_key_id = <ACCESS KEY> secret_access_key = <SECRET KEY> endpoint = <NAMESPACE>.compat.objectstorage.<REGION>.oraclecloud.com
Note that if the storage bucket you created is not in your home region, you may also need to add this entry to the [myobjectstorage] stanza:
region = <REGION>
You are now ready to start copying files to object storage. The following copies a file, myfile.txt to object storage. You can show the contents of object storage using rclone ls.
$ echo `date` > myfile.txt $ rclone copy myfile.txt myobjectstorage:/mybucket $ rclone ls myobjectstorage:/mybucket
Rclone is a useful command line utility to interact with, among other types, S3 compatible cloud-based object storage. Oracle Cloud Object Storage has an S3 compatible API. In this blog post, I showed how to install Rclone from Oracle Linux yum server and configure it using free Oracle Cloud Object Storage.