We recently released a new deployment type in OCI GoldenGate called Data Transforms. This new deployment type adds batch ETL/ELT processing capabilities to the OCI GoldenGate unified data fabric platform. 
In this second technical article in the Get Started with OCI GoldenGate Data Transforms series, we will focus on connectivity and describe how you can connect to your sources and targets before developing data integration processes.

OCI GoldenGate Network configuration

First, we will cover how connectivity works in OCI GoldenGate. This part applies to all deployment types (Data replication, Data transforms, and Stream analytics). 

OCI GoldenGate deployments are located in a secure area in OCI outside your tenancy. Its network isn’t connected to any customers’ networks and can only access resources available over the public internet by default. 

Deployment access

When creating a deployment, you must select a private subnet used to create a private endpoint. It is displayed on the Deployment details as the Private IP address. By default, OCI GoldenGate deployments can only be accessed from a machine with access to your selected private subnet.

Optionally, you can enable public access to the deployment. If so, a Public IP address will also be displayed on the Deployment details page, as shown in the screenshot below.
Data Transforms Deployment Details 
These two quickstarts describe how to access a deployment:

Source/Target Access

Connections must be created and assigned to a deployment to enable connectivity from the deployment into the source/target technology. 

When you create a Connection, you select a traffic routing method:

  • Shared endpoint routes traffic through a private endpoint shared across all Connections set to use Shared endpoint and assigned to the same deployment. When Connections using a Shared endpoint are assigned to a deployment, Ingress IPs are listed on the Deployment details page and in the Assigned deployments table of the Connection details page.
    • Note that shared endpoint connections share bandwidth, which may impact performance.
      Data Transforms Shared Endpoint Connection
  • Dedicated endpoint routes traffic through a single dedicated private endpoint created in the selected subnet. We recommend choosing a private subnet whenever possible, as a NAT Gateway is usually required to access private or public resources. Ingress IPs for dedicated endpoint connections are listed on the Assigned Connections page of the associated Deployment and Connection details pages.

Data Transforms Dedicated Endpoint Connection
 

Note: 

  • The traffic from OCI GoldenGate will originate from these Ingress IPs. You must ensure appropriate subnet security and route rules are in place to allow connectivity into the data source or target node’s IP.
  • The traffic from OCI GoldenGate will always traverse through your VCN. Connections don’t have a public IP, so this VCN must have a NAT Gateway and the appropriate route rules to route traffic to the internet through the NAT Gateway. 
  • Any Fully Qualified Domain Name (FQDN) provided at the Connection level must be resolvable within the selected subnet.

Refer to the OCI GoldenGate connectivity page in our documentation for more information.

Create Connections for OCI GoldenGate Data Transforms

You must create a Generic connection for any connection you intend to use with OCI GoldenGate Data Transforms. As mentioned above, Connections are required to enable connectivity from the OCI GoldenGate deployment.

To create a Generic connection, go to GoldenGate in OCI Console, click Connections, and click Create connection.

Enter a Name and select Generic connection as the Type. 
Create a GoldenGate Connection in OCI Console 
Click Next. Enter the host and port as host:port, and select the Traffic routing method: Shared or Dedicated endpoint.
Generic Connection for ADB 
Click Create.

Once created, you can assign the Generic connection to your Data transforms deployment. You can click Test Connection to check if the deployment can connect.

Generic Connection Examples

Here are some examples of host:port values to use when creating Generic connections for OCI GoldenGate Data Transforms.

  • Autonomous Data Warehouse / Autonomous Transaction Processing
    • Enter adb.<region>.oraclecloud.com:1522 for Autonomous Databases Serverless instances
    • Ex: adb.us-phoenix-1.oraclecloud.com:1522 for an instance running in US West (Phoenix)
  • OCI Object Storage
    • Use the Swift endpoint to connect: swiftobjectstorage.<region>.oraclecloud.com:443. 
    • Ex: swiftobjectstorage.us-phoenix-1.oraclecloud.com:443 to access Object Storage buckets in US West (Phoenix)
  • GoldenGate deployments
    • Enter the GoldenGate deployment FQDN and port 443
    • Ex: yojy…d3ra.deployment.goldengate.us-phoenix-1.oci.oraclecloud.com:443.
  • PostgreSQL
    • Provide either the database FQDN or its IP address and its port. If you enter an IP address, it will be automatically converted into an FQDN, which must be used when creating the connection in Data transforms.
      • Ex: 129.146.69.20:5432 will be converted into ip-129-146-69-20.ociggsvc.oraclevcn.com:5432.
  • Snowflake
    • Enter the host as <account name>.<region>. snowflakecomputing.com and port 443.
    • Ex: kq..60.us-west-2.snowflakecomputing.com:443

Create Connections in Data Transforms deployment

In the current OCI GoldenGate Data Transforms release, you must create a Connection inside the Data Transforms deployment.

Go to your Data Transforms deployment, click Launch Console, and log into the deployment.
Launch Console for Data Transforms deployments 
Once you are logged in, click Connections and click Create Connection.
Create Connection in Data Transforms

Select the technology and click Next to configure the Connection. For example, pick Oracle to connect to an Autonomous Database, an Oracle Database, or an Exadata instance running on OCI or on-premises. 

Click Test Connection to verify you can connect.

Refer to the Data Transforms documentation for Connections for more information.
You can now start designing Data Loads, Data Flows, and Workflows: Create a Data Load, Create a Data Flow, and Create a New Workflow. We will cover this in the next blog in the Get Started with OCI GoldenGate Data Transforms series.

Next steps

Try it out! OCI GoldenGate Data Transforms is available today. Read the documentation to explore more features and benefits. You can also go through the OCI GoldenGate Data Transforms quickstart to familiarize yourself with the service capabilities.

More resources: