Hello, my name is Pinkesh Valdria, and I'm a Solutions Architect working on Big Data for Oracle Cloud Infrastructure.
This post is a follow-up to our post about deploying Confluent on Oracle Cloud Infrastructure Compute instances. Now you can use Terraform automation to deploy Confluent Platform using Helm charts on Oracle Cloud Infrastructure Container Engine for Kubernetes.
Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE) is a fully-managed, scalable, and highly available service that you can use to deploy your containerized applications to the cloud. Use Container Engine for Kubernetes when your development team wants to reliably build, deploy, and manage cloud-native applications. You specify the compute resources that your applications require, and Container Engine for Kubernetes provisions them on Oracle Cloud Infrastructure in an existing tenancy.
Figure 1: OKE High-Level Architecture
Confluent Open Source brings together the best distributed streaming technology from Apache Kafka and takes it to the next level by addressing the requirements of modern enterprise streaming applications. It includes clients for C, C++, Python, and Go programming languages; Connectors for JDBC, Elasticsearch, and HDFS; Confluent Schema Registry for managing metadata for Kafka topics; and Confluent REST Proxy for integrating with web applications.
Figure2: Confluent Platform Components
Helm is an open source packaging tool that helps you install applications and services on Kubernetes. Helm uses a packaging format called charts, which are a collection of YAML templates that describe a related set of Kubernetes resources.
The Terraform automation template performs the following steps:
This Terraform template is available on our cloud partner repository.
Figure 3: Cluster with Node Pools
By default, the Confluent Helm chart deploys three pods for Zookeeper, three pods for Kafka, and one each for Schema Registry, Kafka Rest, Kafka Connect, and KSQL. Grafana and Prometheus monitoring are optional.
Figure 4: Pods Deployed on Clusters
We hope that you are as excited as we are about Confluent Platform deployment on Container Engine for Kubernetes. Let us know what you think!
Principal Solutions Architect, Big Data