OCI Service Operator for Kubernetes

OSOK is based on the Operator Framework, an open source tool used to manage operators. It uses the Kubernetes controller-runtime library, which the Operator SDK uses to provide high-level APIs and abstractions to write operational logic and provides tools for scaffolding and code generation. OSOK is an official project built and maintained by the OCI Container Engine for Kubernetes (OKE) team. We plan to continue investing in this project and working with teams across OCI to expand the number of supported services.

How do I use OSOK?

First time users begin by installing the Operator SDK onto the machine used to connect to your Kubernetes clusters. The Operator SDK is a framework that uses the controller-runtime library to make writing operators easier.

Next, configure the appropriate OCI Identity and Access Management (IAM) service policies to allow OSOK to manage OCI services in your tenancy. You can configure these policies to support deployments of OSOK both in and outside of OCI and limit which services OSOK can control by only including permissions to manage specific services in the policy.

Following the configuration of appropriate policies, we can install the OSOK Operator Lifecycle Manager (OLM) bundle. OLM extends Kubernetes to provide a declarative way to install, manage, and upgrade Operators and their dependencies in a cluster. OSOK is packaged as an OLM bundle to make it easy to install in Kubernetes clusters. The OSOK OLM bundle contains all the required details, such as CRDs, RBACs, configmaps, and deployments, which install the OSOK in the Kubernetes cluster. 

A successful installation of OLM returns the following output:

$ operator-sdk olm status
INFO[0008] Fetching CRDs for version "0.18.3"
INFO[0008] Fetching resources for resolved version "v0.18.3"
INFO[0033] Successfully got OLM status for version "0.18.3"

After downloading the bundle’s docker image, run it using the Operator SDK:

$ operator-sdk run iad.ocir.io/oracle/oci-service-operator-bundle:1.0.0
INFO[0030] Successfully created registry pod: iad-ocir-io-oracle-oci-service-operator-bundle-1-0-0
INFO[0031] Created CatalogSource: oci-service-operator-catalog
INFO[0034] OperatorGroup "operator-sdk-og" created
INFO[0035] Created Subscription: oci-service-operator-v1-0-0-sub
INFO[0036] Approved InstallPlan install-trllz for the Subscription: oci-service-operator-v1-0-0-sub
INFO[0036] Waiting for ClusterServiceVersion "default/oci-service-operator.v1.0.0" to reach 'Succeeded' phase
INFO[0037]   Waiting for ClusterServiceVersion "default/oci-service-operator.v1.0.0" to appear
INFO[0050]   Found ClusterServiceVersion "default/oci-service-operator.v1.0.0" phase: Pending
INFO[0052]   Found ClusterServiceVersion "default/oci-service-operator.v1.0.0" phase: Installing
INFO[0070]   Found ClusterServiceVersion "default/oci-service-operator.v1.0.0" phase: Succeeded
INFO[0071] OLM has successfully installed "oci-service-operator.v1.0.0"

After installing OSOK, you can upgrade and delete it using the operator SDK.

Verify a successful installation by checking for the new CRDs with the following command:

$ kubectl get crds
NAME                                          CREATED AT
autonomousdatabases.oci.oracle.com            2021-08-26T17:16:41Z
catalogsources.operators.coreos.com           2021-08-26T17:01:59Z
clusterserviceversions.operators.coreos.com   2021-08-26T17:02:01Z
installplans.operators.coreos.com             2021-08-26T17:02:02Z
mysqldbsystems.oci.oracle.com                 2021-08-26T17:16:39Z
operatorconditions.operators.coreos.com       2021-08-26T17:02:04Z
operatorgroups.operators.coreos.com           2021-08-26T17:02:05Z
operators.operators.coreos.com                2021-08-26T17:02:06Z
streams.oci.oracle.com                        2021-08-26T17:16:39Z
subscriptions.operators.coreos.com            2021-08-26T17:02:08Z/pre>

Now that OSOK is installed, you can use it to create, update, and delete OCI services and bind them to existing services.

The MySQL DB System controller

One of the supported services at launch is the OCI MySQL Database service. This service allows you to provision MySQL DB Systems with tasks like backup, recovery, and database and operating system patching fully managed by OCI. You are responsible solely for managing data, schema designs, and access policies. OSOK users with permission to manage the mysql-family resource type can create a YAML file, specifying parameters of the mysqldbsystems custom resource. This parameter includes a display name and compartment ID, a choice of the underlying Compute shape and data storage size, and other optional values.

You can also use a Kubernetes secret to securely pass in the admin username and password for the database. After providing those parameters, you can apply the YAML file as you would any other Kubernetes manifest, which triggers the OSOK MySQL DB System controller to automatically provision an MySQL DB System based on the provided specifications. OSOK also allows you to bind to an existing MySQL DB System. In this case, the OCID of the existing MySQL DB System is the only required field in the custom resource spec. You can also update the MySQL DB System.

To demonstrate OSOK, I created a mysqldb.yaml file, including the mandatory parameters listed, and a secret with the database admin username and password. I applied the YAML file, which created a custom resource called osokmysql in my cluster and provisioned a MySQL DB System in OCI based on my parameters.

$ kubectl apply -f mysqldb.yaml
secret/admin-password created
mysqldbsystem.oci.oracle.com/osokmysql created

I can use kubectl get mysqldbsystems to list my mysqldbsystems custom resources, including resources in the process of being provisioned. I can use kubectl describe mysqldbsystems osokmysql to see the events of my MySQL DB System being created. My new OSOK-MySQL DB System is also visible in the OCI Console.

After a few minutes, I checked my mysqldbsystems custom resources again and found that they were successfully provisioned.

$ kubectl get mysqldbsystems -o wide
NAME        DISPLAYNAME   STATUS   OCID                                                                                       AGE
osokmysql   OSOK-MySQL    Active   ocid1.mysqldbsystem.oc1.iad.aaaaaaaal3sy43wcfd277i53ahdm4vpdkxnzfoiut2a5nyz5qyccbqdatc7q   57m

I used kubectl describe mysqldbsystems osokmysql to show the events related to this custom resource, which you can use to confirm that the resource was successfully created.

Events:
  Type    Reason   Age                From           Message
  ----    ------   ----               ----           -------
  Normal  Success  58m                MySqlDbSystem  Finalizer is added to the object
  Normal  Success  50m (x2 over 50m)  MySqlDbSystem  Create or Update of resource succeeded

I verified on the OCI Console too.

After the resource is created, access information used to manage the database is created as a Kubernetes secret. This information varies depending on the type of resource. For MySQL, it includes DNS endpoint, MySQL port, private IP address, endpoints to connect to the MySQL DB System, and more. All this information is accessible directly from Kubernetes.

$ kubectl describe secret osokmysql
Name:         osokmysql
Namespace:    default
Labels:       <none>
Annotations:  <none>
​
Type:  Opaque
​
Data
====
PrivateIPAddress:    11 bytes
AvailabilityDomain:  20 bytes
Endpoints:           136 bytes
FaultDomain:         14 bytes
InternalFQDN:        0 bytes
MySQLPort:           4 bytes
MySQLXProtocolPort:  5 bytes
</none></none>

Why would I use OSOK?

OSOK is useful for production Kubernetes users looking to simplify the lifecycle management of services outside their Kubernetes cluster. Not only can you take advantage of familiar Kubernetes configuration management, but also you can use Kubernetes tooling. As we saw in the example, I can use Kubernetes commands, such as kubectl apply, get, and describe, to track resources outside my Kubernetes cluster. You can also use RBAC for access control and events for visibility.

OSOK can also simplify the testing and development process by making it easy to deploy and delete resources. Developers no longer need to use a separate set of OCI APIs to create managed resources outside their cluster. OSOK makes it easy to provision them using a Kubernetes manifest. OSOK also makes it easy to connect to existing resources, such as a mock customer data dataset stored in an Autonomous Data Warehouse test database.

Wrapping up

We built the OCI Service Operator for Kubernetes (OSOK) to remove friction for Kubernetes users looking to incorporate managed OCI services into their container architecture. We chose to build a tool that allows users to use the familiar Kubernetes configurations and tooling they’re already comfortable with instead of expecting users to familiarize themselves with the OCI developer tools or asking them to manage yet another set of infrastructure.

Today, OSOK supports the Autonomous Database service, the MySQL Database service, and the OCI Streaming service. In the coming months, we plan to expand the number of supported OCI services.

You can learn more about how to get started installing and using OSOK from the Adding OCI Service Operator for Kubernetes to Clusters documentation and from our page on GitHub. Feel free to reach out with feedback or make your own contributions to the OSOK project. We hope that OSOK is an example for people looking to build their own operators.