Replication Paths between Oracle GoldenGate MSA Marketplace

August 12, 2021 | 8 minute read
Sydney Nurse
EMEA AppDev & Security Expert
Text Size 100%:

Replication Paths between Oracle GoldenGate MSA Marketplace

Microservices Architecture for Non-Oracle!

In recent weeks we've seen the release of Oracle GoldenGate (OGG) version 21.1.0 and the arrival of Microservices Architecture (MSA) for Oracle and heterogeneous databases:

  • Oracle Database 21c
  • DB2 z/OS
  • MySQL
  • SQL Server
  • Big Data

Oracle GoldenGate Microservices Architecture

The improved user experience and additional performance monitoring options are now available for the main stream Sources and Targets for replication. The main focus however for this article will be on establishing replication distribution / receiver paths between instances, in particular the MarketPlace images and where self-signed certificates are provided by default, have been notoriously difficult to manage.

Oracle Wallets

Oracle Wallet is a container/repository that stores credentials such as certificates, certificate requests, and private keys. OGG MSA stores user credentials and certificates in wallets accessible for each deployment.

Oracle Wallets provide the basis of secure GoldenGate deployments and Restful API calls between the Distribution and Receiver services over SSL/TLS, storing the server and client certificates. Orapki was used in previous MSA deployments to both create and manage certificates via the command line (CLI). OGG MSA 21.1 also delivers Certificate management via the Web UI with support for multiple client certificates for wss protocol.

This new feature was my top highlight of this release, above MSA for additional technologies and support for Azure SQL Server. FOr the Big Data option the Dependency Downloader is a notable new feature in the Big Data release notes. (*It should be noted that officially from this release that Oracle Database OGG Classic Extract is now de-supported and Oracle Database OGG Classic Architecture is marked as deprecated.*)

OGG MSA Marketplace Certificates

Cloud deployments should be easy, simply and fast, more or less and the Cloud Marketplace is a great option for capabilities that have no service equivalent or for customers seeking more control while still taking advantage of cloud infrastructure benefits. The OGG Marketplace image has been available for some years now, providing fast simple deployments for goldenGate customers and can be used with both classic and MSA deployments on premise or in Oracle or 3rd party clouds.

The complete OGG MSA is implemented, fully functioning and ready to be configured for your replication scenario. Each provisioned instance has a secured deployment using self-signed certificates. In order to establish communication between MSA instance, we need to either exchange the default certificates with our own or copy and import them between instances.

The rest of this article covers the latter; easiest means to location and copy the certificates and then importing them into the deployment. In my case, I have taken the target deployment certificates and imported them into my source deployment.

Which Certificates from which Deployment?

For a typical Distribution Server Path to a Receiver, the Receiver Server certificates will be required on the Distribution side. You can think of the Distribution Server of the Source Deployment and the client that wishes to connect to the APIs provided by the Receiver Server on the Target Deployment.

Client Certificate (Public and Certificate Chain)

Oracle GoldenGate Marketplace images are based on Oracle Linux Server with 2 main mount points, as far as OGG MSA is concerned; /u01 and /u02. The binaries are located on /u01 and deployment(s) on /u02 where we will find the location of the wallets. The wallets will contain the client certificate, required certificate chains and Certificate Authority or Root CA cert. All of these are required to validate the client connection and establish trust for the certificate.

Access to the wallet requires a password, which is not known to us, so export and import using CLI orapki is not possible. No fear, a far more easy option is available via Firefox. Google Chrome and Safari require easing security restrictions to access site that use self-signed certificates, so for now the recommendation is Firefox. Follow the steps to provision and configure the instance, including and post provisioning steps for 3rd Party libraries etc. and start the Admin Server process.

Once provisioning is complete, from Firefox, connect to the OGG MSA provided IP over https. Accept the Risks of connecting with GoldenGate and login with the generated oggadmin credentials from the instance. For now, the only step is to successfully connect as we will use the default browser capability to view and download our client certificate and chain to our local machine. The client certificate is the public portion of the certificate but we will also require the private key to complete the import process.

From the browser Site warning and details, navigate through to display the certificate and certificate chain.

The last certificate that is required is located on the OGG MSA instance itself, the Private Key which is stored along with the nginx server configuration. Using SSH, connect to the instance, and download the private key file, ogg.pem, from /etc/nginx/ onto your local machine.

You should now have 3 files named similar to these:

Client Cert cnsesnggora01-app-odicnfjbzlu-oraclevcn-com.pem
Client Cert Chain cnsesnggora01-app-odicnfjbzlu-oraclevcn-com-chain.pem
Private Key Cert ogg.pem

 

Add the Client and CA Certificates

The above certificates need to be added to the Source Deployment via the Service Manager, though I have not found the documentation covering this feature addition on Oracle Docs website. Lucky for us OGG MSA also provides a fairly complete set of embedded help information. Once you've navigated to the certificate management page, select the Help "?" icon on the top right and then choose

Certificates

Use this tab to manage certificates for client and CA certificates. See Certificate Management for details.

On the main Service Manager page, there are two ways to navigate to Certificate Management, the Navigation Menu, top left or via the managed deployments listed in the lower portion of the site. Selecting the menu item: Certificate Management or Deployment name and the Certificates tab, will display the current certificates. 

The steps are simple, click the plus (+) sign to Add Client Certificates

Enter the following details for the client certificate and clid Add:

  • Unique Name: Name of the certificate.
  • Certificate PEM: Enter a certificate .pem file or upload a .pem file.
  • Private-Key PEM: Enter or upload the private key for the .pem file.
  • CA Certificates: Enter or upload the CA certificate.

Then click the plus (+) sign to Add CA Certificates. (Yes, CA Certificates need to be added twice)

Enter the following details for the CA certificate and clid Add:

  • Unique Name for the CA certificate.
  • Certificate PEM value can be entered in the box or uploaded.
  • Certificate Location can be shared. CA Certificates are always shared and cannot be local. When adding or replacing CA certificates, the Shared option is always force-checked.

That's it!

Creating a Replication Paths

A replication path contains the connection and trail file, source and remote, details. For MSA deployments, an Alias is also needed, regardless if the connection is unsecured websocket or secured SSL/TLS. Certificates are required for secured communications i.e. secured websockets (wss)

Which User, Credential and in which Deployment should they be Created?

This is similar as with the certificates, where the credential is referenced by the Distribution Server on the Source Deployment with an Alias that stores the Target Deployment user details. From the image above, you can see where each user object is created. For configurations requiring a Reverse Path, i.e. the Path is created on the Receiver Server, then everything discussed above would be reversed as well. 

Create Your Path

The last steps are to create the path between the Source and Target deployments. Extracts and trail definitions should be completed and successfully tested. In some cases the replicat on the Target may need to be defined as well. Creating the path is well documented but I would like to highlight some items and make some suggestions.

By default, source and target URIs are built up based on the values provided, say hostname, port, trail etc. For connections between instance on OCI, use the Private IP by adding a record in each host instance that will be apart of the replication scenario. In my instance I've adding host entries to the /etc/hosts file on each instance. For the Source, the default is the Internet facing Public IP, if this is the case, edit the generated Source URI to use localhost, the host name, FQDN or Private IP directly.

The Path will be created on the Target Receiver side and reference the Public IP. I am not sure if this will be used for acknowledgements of messages received, so I have always updated this property.

Updating / Modifying Path Properties

Only a limited set of properties can be updated or modified once the path has been saved. Most important values like Source and Target URI, Extract and Trail, can not be changed. To correct any value other than the Domain and Alias used for the connection, the Path will need to be deleted and re-created. I am postive the configuration is stored in json under the covers in the deployment dat file but this is a bit risky for what should amount to a minute's worth of work.

Summary

The Marketplace image provides several advantages but is pre-configured with self-signed certificates. This is okay for testing and development and connecting OGG MSA deployed with the Marketplace images is a piece of cake, with a little bit of guidance on the location of the certificates. If you like cake :)

In summary, OGG 21.1 has a lot to offer and delivers the Web UI to Oracle, Oracle, IBM DB2 z/OS, Microsoft SQL Server and Big Data. For those of you thinking I've made some kind of typo Oracle2, think about all the great technologies that Oracle owns and that GoldenGate supports. 

Certificate management over the Web UI is fantastic and eases all the work required around Oracle Wallets.

And lastly, consider which deployment will start the connection vs. which deployment is providing the OGG Microservice API, when collecting certificates and establishing users or user aliases.

 

 

 

Sydney Nurse

EMEA AppDev & Security Expert

One thing Golf teaches you, is attention to detail, practice to execution and trust.

These same principals apply to expertise in an given area, and even more so in Software Systems.

 

Show more

Previous Post

Pivoting data in Oracle Cloud Infrastructure Data Integration

Basavaraja Allundi | 4 min read

Next Post


Oracle GoldenGate 21c Release Announcement

Mack Bell | 2 min read
Oracle Chatbot
Disconnected