Connecting OCI GoldenGate to Confluent Cloud Private Endpoints

Oracle Cloud Infrastructure (OCI) GoldenGate is a fully managed, native cloud service that moves data in real-time, at scale. On top of core GoldenGate for Big Data capabilities, it brings additional advantages of a fully managed service like auto-scaling, flexible/ data throughput-based licensing, improved user experience. 

In this blog, we will cover the required settings for configuring an OCI GoldenGate Confluent Replication targeting Confluent Cloud running with private endpoints. If interested, you can refer to below resources about OCI GoldenGate and Confluent:

• Replicate Data from Autonomous Transaction Processing to Confluent Kafka 
• OCI GoldenGate Confluent Replication
• OCI GoldenGate Documentation

This blog post covers the steps for configuring the connectivity between OCI GoldenGate and Confluent Cloud running with Private Endpoints.

Prerequisites:

Please note that, you may need to work with your networking and Confluent teams to complete the following prerequisites.

1. Replicating into Confluent Cloud Private Endpoints with OCI GoldenGate requires IPSec VPN or interconnect between OCI and target cloud service provider. Please make sure that connectivity is successful before moving into next steps.
2. After IPSec VPN/ interconnect is in place, please configure Confluent Cloud Private Endpoints and DNS settings in target cloud service provider. 
   For configuring Confluent Cloud Private Endpoints, you can refer to following Confluent documents: 
   • Azure Private Link with Confluent Cloud
   • AWS Private Link with Confluent Cloud
   • Google Cloud Pirvate Service with Confluent Cloud
3. After configuring DNS settings in target cloud service provider, configure DNS settings in OCI. In OCI, you can configure the DNS records in Networking/ DNS Management/ Private Views. Please make sure that you use the Private View named with your VCN name which is connected to target 3rd party cloud.
4. After configuring the network and DNS records on both OCI and target cloud, you can test the connection from a compute VM running in OCI with the subnet which is connected to your target cloud. It is reccommended to use a kafka client producer to publish messages into target cluster.

Create OCI GoldenGate Connections:

We need to create a Confluent Cloud connection. For the detailed steps, you can refer to OCI GoldenGate Documentation. 

Shared endpoint traffic routing method will route the traffic from through the same subnet as the deployment subnet.

As the host, use the bootstrap server and port as 9092. You don't need to add any private ip. When customer assigned subnet is selected, OCI GoldenGate will perform DNS lookups in the private subnet selected. For Security protocol, select SASL over Plaintext. Use api key/ secret as username/ password.

After connection created and assigned to deployment, it will get ingress IPs. Please make sure that they're whitelisted in your deployment's subnet security list.

After assigning the connection to your deployment, you can create a connection for Confluent Schema Registry if needed. And you can confgiure the replicat.