We are proud to announce a validated reference architecture for Confluent Platform on Oracle Cloud Infrastructure. Starting today you can deploy Confluent's industry-leading distributed streaming platform on Oracle's high-performance cloud by using Terraform templates.
With this announcement, Oracle Cloud Infrastructure enhances its Big Data ISV ecosystem of partners. The partnership between Confluent and Oracle enables you to connect all your interfaces and data systems so that you can make decisions leveraging all your internal systems in real time, all on a unified enterprise cloud platform with unmatched performance, security, and availability.
The Confluent Enterprise flavor of Confluent Platform brings together the best distributed streaming technology from Apache Kafka and addresses the requirements of modern enterprise streaming applications. Confluent Enterprise includes the following components:
Figure 1: Confluent Platform Components
Oracle offers the most powerful bare metal compute instances with local NVMe flash storage in the industry. Only Oracle offers this local storage, based on advanced NVMe SSD technology, and backed by a storage performance SLA.
The bare metal compute instances are connected in clusters to a non oversubscribed 25-gigabit network infrastructure, guaranteeing extremely low latency and very high throughput, which is a key requirement for high-performance distributed streaming workloads. In fact, Oracle Cloud Infrastructure is the only cloud, with a network throughput performance SLA. Oracle Cloud Infrastructure also supports virtual machines offerings for compute instances
Confluent clusters that are spun up in the cloud can sit right next to your favorite Hadoop/Spark clusters using Cloudera, Hortonworks, or Mapr, and also next to Oracle’s Database environments—Oracle Autonomous Data Warehousing or Oracle Autonomous OLTP services. With Confluent Connectors to connect Apache Kakfa to other data systems such as Oracle Cloud Object Storage, Apache Hadoop, JDBC, Elasticsearch, Cassandra, IBM MQ, it allows for easy data sharing for analytics, monitoring and more. Integrating Confluent with Oracle Object Storage is very simple & quick using Kafka Connect S3 connector because Oracle Object Storage supports Amazon S3 Compatibility API. This ensures our customers are not being locked into a single vendor storage service and it gives them the ability to continue using their favorite client, application, or service with Oracle Object Storage. For details on how to integrate Confluent with Oracle Object Storage, see the Readme.md file.
Cloud infrastructure enables you to deploy the optimal amount of infrastructure to meet your demands. No more under-utilization of too much infrastructure, or higher latency due to under-forecasting. In addition, Oracle offers:
“This release of the validated reference architecture allows customers the freedom of choice to run Confluent Platform on Oracle Cloud Infrastructure to experience the performance and SLA delivered by Oracle’s bare metal instances and local NVMe platform. Now, Confluent can be deployed right next to Oracle Autonomous DB, enabling customers to unify their data silos and react in real time to events by using a modern scalable event streaming platform powered by Apache Kafka,” said Simon Hayes, Vice President of Corporate and Business Development at Confluent.
You can deploy Confluent Platform on Oracle Cloud Infrastructure by using Terraform automation, which is becoming the leading cross-cloud framework for infrastructure as code (IaC). Choose on of the following Terraform templates:
Figure 2: Multiple-Availability-Domain Architecture
Confluent Platform topology supports three classes of service nodes. You specify three or more broker nodes and one or more worker nodes. A zookeeper quorum is required for metadata management, and it can be deployed on independent nodes or broker nodes. Secondary services such as Confluent Schema Registry and Confluent REST Proxy are deployed on the worker nodes.
To customize your Terraform deployment, you can perform the following actions:
You can deploy both flavors of Confluent Platform: Confluent Enterprise or Confluent Open Source. For details about the Terraform templates, see the Readme.md file.
We hope you are excited as we are about the Confluent Platform on Oracle Cloud Infrastructure solution. Let us know what you think!
Principal Solutions Architect, Big Data