The latest cloud infrastructure announcements, technical solutions, and enterprise cloud insights.

Confluent Platform Now Validated on Oracle Cloud Infrastructure

Pinkesh Valdria
Principal Solutions Architect

We are proud to announce a validated reference architecture for Confluent Platform on Oracle Cloud Infrastructure. Starting today you can deploy Confluent's industry-leading distributed streaming platform on Oracle's high-performance cloud by using Terraform templates.

With this announcement, Oracle Cloud Infrastructure enhances its Big Data ISV ecosystem of partners. The partnership between Confluent and Oracle enables you to connect all your interfaces and data systems so that you can make decisions leveraging all your internal systems in real time, all on a unified enterprise cloud platform with unmatched performance, security, and availability.

Confluent Enterprise Provides a More Complete Distribution of Apache Kafka

The Confluent Enterprise flavor of Confluent Platform brings together the best distributed streaming technology from Apache Kafka and addresses the requirements of modern enterprise streaming applications. Confluent Enterprise includes the following components:

  • Confluent Control Center, for end-to-end monitoring and management
  • Confluent Replicator, for managing multiple-data-center deployments
  • Confluent Auto Data Balancing, for optimizing resource utilization and easy scalability
  • Clients for C, C++, Python, and Go programming languages
  • Connectors for JDBC, Elasticsearch, HDFS, etc.
  • Confluent Schema Registry, for managing metadata for Kafka topics
  • Confluent REST Proxy for integrating with web applications

Figure 1: Confluent Platform Components

Learn more about Confluent Enterprise

Oracle Cloud Infrastructure Provides Big Data Flexibility and Performance

Blazing Fast Performance

Oracle offers the most powerful bare metal compute instances with local NVMe flash storage in the industry. Only Oracle offers this local storage, based on advanced NVMe SSD technology, and backed by a storage performance SLA.

The bare metal compute instances are connected in clusters to a non oversubscribed 25-gigabit network infrastructure, guaranteeing extremely low latency and very high throughput, which is a key requirement for high-performance distributed streaming workloads. In fact, Oracle Cloud Infrastructure is the only cloud, with a network throughput performance SLA. Oracle Cloud Infrastructure also supports virtual machines offerings for compute instances

Unmatched Data Ecosystem

Confluent clusters that are spun up in the cloud can sit right next to your favorite Hadoop/Spark clusters using Cloudera, Hortonworks, or Mapr, and also next to Oracle’s Database environments—Oracle Autonomous Data Warehousing or Oracle Autonomous OLTP services. With Confluent Connectors to connect Apache Kakfa to other data systems such as Oracle Cloud Object Storage, Apache Hadoop, JDBC, Elasticsearch, Cassandra, IBM MQ, it allows for easy data sharing for analytics, monitoring and more. Integrating Confluent with Oracle Object Storage is very simple & quick using Kafka Connect S3 connector because Oracle Object Storage supports Amazon S3 Compatibility API. This ensures our customers are not being locked into a single vendor storage service and it gives them the ability to continue using their favorite client, application, or service with Oracle Object Storage. For details on how to integrate Confluent with Oracle Object Storage, see Oracle Object Storage and Confluent Connector.md file.


Right-Size Your Infrastructure in the Cloud

Cloud infrastructure enables you to deploy the optimal amount of infrastructure to meet your demands. No more under-utilization of too much infrastructure, or higher latency due to under-forecasting. In addition, Oracle offers:

  • The lowest compute pricing from a pay-as-you-go (PAYG) perspective
  • The lowest network egress costs in the industry

This release of the validated reference architecture allows customers the freedom of choice to run Confluent Platform on Oracle Cloud Infrastructure to experience the performance and SLA delivered by Oracle’s bare metal instances and local NVMe platform. Now, Confluent can be deployed right next to Oracle Autonomous DB, enabling customers to unify their data silos and react in real time to events by using a modern scalable event streaming platform powered by Apache Kafka,” said Simon Hayes, Vice President of Corporate and Business Development at Confluent.

Deploying Confluent Platform

You can deploy Confluent Platform on Oracle Cloud Infrastructure by using Terraform automation, which is becoming the leading cross-cloud framework for infrastructure as code (IaC). Choose one of the following Terraform templates:

  • community, which deploys Community edition of Confluent Kafka in a single availability domain across 3 fault domains
  • enterprise, which deploys Enterprise edition of Confluent Kafka in a single availability domain across 3 fault domains

Figure 2: Multiple-Availability-Domain Architecture

Confluent Platform topology consist of Zookeepers(minimum: 3 nodes), Brokers(minimum: 3 nodes), Schema Registry, REST Proxy, Kafka Connect, KSQL and Control Center nodes.  

To customize your Terraform deployment, you can perform the following actions:

  • Choose the version and edition of Confluent Platform to deploy.
  • Configure the number and shape for various nodes to deploy (zookeeper, broker, connect, rest, schema registry, etc.)
  • Specify storage capacity for broker instances.
  • Change CIDR block sizes and other configuration settings.

For details about the Terraform templates, see the Readme.md file.

What’s Next?

  1. If you don’t have an Oracle Cloud Infrastructure account yet, you can sign up for a 30-day free trial account.
  2. Follow the instructions on the GitHub oci-confluent page to install Confluent Platform on Oracle Cloud Infrastructure

We hope you are excited as we are about the Confluent Platform on Oracle Cloud Infrastructure solution. Let us know what you think!

Pinkesh Valdria

Principal Solutions Architect, Big Data


Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha