Last week we packaged up a few more RPMs for Oracle Linux 7 that will help make life easier for Cloud users.
bbcp 15.02.03.01.1-3 in ol7_developer:
# yum install bbcp
bbcp is what I would call ssh on steroids. If you want to copy files from a local node to a remote node (say in Oracle Cloud) then this is a great tool. It might require some tuning but the idea is that you can open up parallel TCP streams. When you do large file transfers this should be able to give you a bit of a performance boost. I would also recommend using UEK5 and enable BBR as the congestion control algo. (see an old blog entry). The combination of enabling BBR (only has to be done on one of the 2 nodes (src or dest)) and using bbcp to copy large files using parallel streams should provide you the best throughput. By making this into an RPM for OL, it makes it easily available for everyone to use.
rclone 1.42 in ol7_developer
# yum install rclone
rclone is a very cool command line tool to move files around from/to local storage and cloud object storage. This works very well with Oracle Cloud Infrastructure's Object Storage. Now that it's packaged as an RPM with OL you can just install it directly from the command line instead of having to go download a file from a website. rclone works like scp.
Example could be # rclone copy localdir ocistorage:remotedir
In order to configure rclone for Oracle Cloud Infrastructure's Object Storage, you have to create an "Amazon S3 Compatible API Key". This generates a secret key that you have to use during rclone config along with the access key (looks like an OCID in Object Storage ocid1.credential.oc1.<string>) .
Configuration example:
# sudo yum install -y rclone
-> In the OCI console you go to Identity -> Users -> User Details -> Amazon S3 Compatible API Key and generate a new Secret Key.
-> copy the secret key because you need that to configure rclone, and you will also need the Access Key (which is an OCID)
-> configure rclone on your OL7 client.
Example :
# rclone config
-> type n (new remote) and give it a name
name> ocistorage
Type of storage to configure.
-> type 3 (Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio))
Choose your S3 provider.
type 8 (Any other s3 compatible provider)
-> Next type 1 (1 / Enter AWS credentials in the next step)
For access key provide the ocid
-> access_key_id> ocid1.credential.....
For the secret access key use your secret key that was just generated.
secret_access_key> tyjXhM7eUuB2v........
Region to connect to.
-> hit enter
For endpoint (example, phoenix) enter a https url
example : https://orclwim.compat.objectstorage.us-phoenix-1.oraclecloud.com
my tenant name is orclwim so replace it with your tenant name.
The end point URLs are
https://<tenantname>.compat.objectstorage.us-phoenix-1.oraclecloud.com
https://<tenantname>.compat.objectstorage.us-ashburn-1.oraclecloud.com
https://<tenantname>.compat.objectstorage.eu-frankfurt-1.oraclecloud.com
https://<tenantname>.compat.objectstorage.uk-london-1.oraclecloud.com
Location Constraint hit enter
and ACL hit enter
type y OK to store the settings
you should get something like
Current remotes:
Name Type
==== ====
ocistorage s3
That's it - we have some code changes pending that will include oracle and the endpoints in rclone but those are being reviewed still.