Top 7 Oracle GoldenGate Announcements at CloudWorld 2023

September 19, 2023 | 11 minute read
Jeff Pollock
Vice President
Text Size 100%:

Top 7 GoldenGate Announcements at CloudWorld 2023

Oracle GoldenGate continues its reign at the forefront of real-time data management with more investments in AI, Cloud, and streaming technologies. Broad industry shifts towards data fabric, mesh architecture, distributed data, and multi-cloud solutions play right into GoldenGate's core strengths and those strengths are why the most successful businesses in the world depend on it as a foundation for their data platforms. GoldenGate is trusted by 1000’s of customers (> 80% of Fortune 500) for operational applications as well as analytic and data science use cases.

With appreciation for our thousands of customers and in anticipation of many more, the GoldenGate team is very happy to announce this year’s major updates to the platform at Oracle CloudWorld. 

Summary of today’s announcements with links for more information:

As you can see the GoldenGate team has had a very busy year!  These announcements are each significant on their own and I encourage you to dive in for more details.  Here is a brief summary of each of these important innovations:

GoldenGate 23c Free

Oracle GoldenGate 23c Free is the same, powerful Oracle GoldenGate that businesses throughout the world rely on. It offers a full-featured experience and is packaged for ease of use and simple download—for free.  Whether you are a developer, a data scientist, a DBA, an educator, or just interested in real-time data, Oracle GoldenGate 23c Free is an ideal way to get started. It provides native support for low-latency replication, modern data types, microservices, and all the top-tier GoldenGate features—all built into one easy to use free package.

This latest release of 23c includes support for Oracle Database 23c and brings a whole new – vastly simplified – user experience for setting up multi-active data replication with Automatic Conflict Detection & Resolution (ACDR).

As always, GoldenGate Free is completely free forever and is easy to download from Oracle Container Registry in its pre-built container.  Read here for more information… 

Innovations for AI, Vector Embeddings and Streaming Data

As one of the hottest areas of tech buzz in 2023, everyone is talking about Generative Artificial Intelligence (GenAI) and the impact it is already having on the world.  Oracle GoldenGate is both being shaped by GenAI as well helping to shape how GenAI can best be used with enterprise data.  Enterprise organizations are rapidly recognizing the benefits of a technique called Retrieval Augmented Generation (RAG) which is a way to incorporate secure, fresh enterprise data into GenAI answers.  Without RAG, training and fine-tuning GenAI models can be costly, insecure and will typically only provide stale data sets.  RAG techniques address this problem by providing fresh data as vector datatype embeddings that can be used with GenAI engines.

The Oracle GoldenGate platform can be used to stream real-time data changes into the RAG vector embeddings used by GenAI engines that answer questions with enterprise data. GoldenGate can be used to prepare and cleanse data in real-time, then ‘vectorize’ the data into the embedding store.  There are several ways to approach this with GoldenGate.

There are 3 things going on in this picture:

  • GoldenGate can capture data events from all popular data producers in real-time and then move those transactions into Oracle Database 23.4 where the change events are transformed to vector embeddings and used with the GenAI program.
  • GoldenGate Stream Analytics can integrate directly with Oracle Machine Learning to invoke external text transformers (eg; user provided) which will then create and store the vector embeddings for further use with Oracle 23.4 vector store.
  • GoldenGate Stream Analytics can use ONNX (Open Neural Network Exchange) integrations to directly interface with any ONNX compatible Large Language Model (LLM).  This process can be used to take any text flowing via the streaming data, vectorize the data and then store into all supported data targets from the streaming pipelines.

GoldenGate Stream Analytics is an ideal platform to prepare, standardize and cleanse data events in real-time prior to the vector embedding process.  This can help to ensure that high quality data makes it into the vector embeddings used by the modern GenAI applications.

Check out Oracle.com/artificial-intelligence for more information about Oracle AI capabilities.

GoldenGate for Distributed Applications and Analytics

Next up, we have an announcement that is one-part name change and two-parts very cool new features.  Let’s cover the name change first!  GoldenGate for Big Data has been one of the most successful independent GoldenGate products of the past decade.  When we launched GoldenGate for Big Data back in 2014, the dominant big data technology platform was Apache Hadoop and Apache Kafka was just starting to take off in popularity. Back then we used to talk about the so-called Lambda architecture for batch and speed processing. 

Today, the data ecosystems are much more diverse in the range of technologies as well as much more distributed in terms of the physical location of data. In the early days of Hadoop, big data was by definition a monolithic and centralized approach. But today’s data producers and data consumers rarely run on the same infrastructure. Systems of record (aka: sources of truth) are themselves often running on a distributed Service Mesh architecture.  Data “hubs” are often just a single node among many other decentralized data repositories. Data Lakehouses rarely serve the needs of an entire enterprise, and instead there are many data lakes with metaphorical data streams and rivers connecting them.

With the above comments as background, we are signaling GoldenGate’s renewed focus on distributed applications and decentralized analytics by renaming “GoldenGate for Big Data” to its new name, “GoldenGate for Distributed Applications and Analytics”.  This new name is a better fit for now and for at least the next decade of modern data solutions. 

We are also announcing two new components to this package, (1) GoldenGate Data Streams and (2) Transaction Manager for Microservices Enterprise Edition (MicroTx EE). Combined with the existing capabilities for (C) Stream Analytics, (D) Capture/Apply from NoSQL and Messaging systems and (E) Apply to dozens of Big Data, Lakehouse and Warehouse target technologies, there is a huge amount of value in this product.  Here is all that is included in the new package:

Now, here is a more detailed look at how these components can be used together:

In case you are reading this as an existing customer of GoldenGate for Big Data, the good news is that you don’t have to do anything to receive the benefits of this new product packaging.  All existing GoldenGate for Big Data customers will be able to leverage the new features and capabilities without any additional licensing or complicated migrations!

Next, let’s learn a little bit more about the new application-centric capabilities:

GoldenGate Data Streams and Microservices Transaction Management

The new product packaging for GoldenGate for Distributed Applications and Analytics is launching two very cool new capabilities that support Application Development and Stream Processing.

  1. GoldenGate Data Streams – a brand new publish-subscribe REST interface to your JSON formatted data events. Leveraging the open AsyncAPI standard, software clients can now subscribe directly to change events from GoldenGate without being forced to stand-up complex and expensive messaging software like Apache Kafka.  Customers can now support “fan out” style data distribution directly from GoldenGate streaming data channels.  Listen to an architectural overview of Data Streams and the AsyncAPI implementation.

  1. Transaction Manager for Microservices Enterprise Edition (MicroTx EE) – MicroTx EE is designed to help application developers ensure a spectrum of data consistency across microservices and cloud native apps without writing any complex code. With a choice of transaction protocols and use of APIs and annotations included in the client libraries, MicroTx EE makes it easy to use distributed transactions in applications deployed in Kubernetes and Docker Swarm.  Developers can easily create multi-phase commits, try-confirm/cancel or long-running Saga transaction patterns across a wide range of Java, Javascript, C/C++, Blockchain and Tuxedo applications.

These exciting new capabilities significantly expand the use cases that GoldenGate can address on the operational and applications side of IT, while remaining true to the heritage of GoldenGate by focusing on helping to ensure high consistency data transactions in distributed data architectures.

Follow this link for a bit more information about the new Oracle GoldenGate for Distributed Applications and Analytics

OCI GoldenGate New Deployments and Connectivity

Oracle Cloud Infrastructure GoldenGate (OCI-GG) was the first fully-managed cloud service for data replication and over the past 12 months the rapid pace of innovation has continued. Last year, we announced OCI GoldenGate Stream Analytics for fully-managed stream processing and that is available now.  This year we are announcing that Data Transforms are coming to OCI GoldenGate. If you are one of the thousands already using Autonomous Database, you may already know about Data Transforms.  Data Transforms has been part of the data tools for Autonomous, and now we are casting a wider net by making it more broadly available for more use cases with OCI GoldenGate!

Data Transforms is a rich data pipeline capability for data transformations, machine learning, and batch/micro-batch ETL/ELT processing. Developers working with OCI GoldenGate can now spin up Data Transforms as a new type of OCI-GG deployment – adding capabilities to work with complex batch data processing regardless of whether an Autonomous DB is in the mix or not.

Last year we also announced support for 30+ new heterogeneous data platforms with OCI GoldenGate.  Now, we are adding more than 40 new data platforms to OCI GoldenGate, for both real-time connectivity as well as from the new Data Transforms deployment type.

With these CloudWorld 2023 announcements, OCI GoldenGate further extends its lead in the cloud data integration space!

Follow this link for more information about the new OCI GoldenGate announcements

OCI Database Migration Updates

Many of you may recall that our team also builds the Oracle Cloud Infrastructure Database Migration service capabilities, providing the fully-managed infrastructure for migrating databases to Oracle Cloud.  This team has been very busy in the past year.  First up, the OCI Database Migration service will make use of OCI GoldenGate for the underlying data replications, this will make it significantly simpler, faster and more reliable for customers who choose to do online migrations. 

Next, we are rolling out capabilities for MySQL database migrations.  The OCI Database Migration service can now support online migrations from a variety of Amazon, Azure, Google and on-premises databases – into Oracle MySQL/Heatwave of course.

Early adopters of this new MySQL functionality have reported significant benefits around scripted automation capability that support fleet migrations, simplified processes that eliminate downloads and MySQL shell usage and plans to migrate 1000’s of databases in groups of 100+ at a time!

Follow this link for more information about OCI Database Migration

Beta Program Launch for Veridata 23c

One more thing!  Veridata is undergoing a massive top-to-bottom transformation and we invite any of our GoldenGate customers to participate in the beta program.  For those that don’t know, Veridata is focused on “data verification” (clever naming!) and provides compare-and-repair functionality for a wide range of different data stores.  Veridata is not specifically dependent on other GoldenGate components and may be used independently.  The main use case for Veridata is to provide trusted and verifiable SLAs that can guarantee data records are the same between data stores.

The next generation of Veridata is being re-written to run on a new microservices back-end architecture (Java Helidon) and it will have a whole new browser-based designer (Oracle JET) to make it simple for a new class of self-service users and more secure cloud-native deployments.  Join us for the beta program and you can give this next-gen Veridata a try for yourself!

Check here for more information about the Veridata beta program launch 

At Cloud World you can learn more

Well, I hope you’ve enjoyed my quick tour of the Top 7 GoldenGate announcements at Oracle CloudWorld!  Our team is thrilled to be able to see you all in person at the event and we can’t wait to hear your feedback on our plans for the future.

If you happen to be joining us in Las Vegas, come see us at the following sessions:

Learn more here:

 

Jeff Pollock

Vice President

Mr. Pollock is an expert technology leader for data platforms, big data, data integration and governance. Jeff has been CTO at successful startups and a senior exec at Fortune 100 tech vendors. He is currently VP of Products and Cloud Services for Data Replication, Streaming Data and DB Migrations. At IBM, he was responsible for all Information Integration and Data Governance products, previously Jeff was an independent architect for US Defense Department, Vice President of Technology at Cerebra and Chief Technical Officer of Modulant – he has been engineering artificial intelligence based data platforms since 2001. As a business consultant, Mr. Pollock was a Head Architect at Ernst & Young’s Center for Technology Enablement. Jeff is also the author of “Semantic Web for Dummies” and "Adaptive Information,” a frequent keynote at industry conferences, author for books and industry journals, formerly a contributing member of W3C and OASIS, and an engineering instructor with UC Berkeley’s Extension for object-oriented systems, software development process and enterprise architecture.

Show more

Previous Post

Connecting OCI GoldenGate to Snowflake Private Endpoints

Deniz Sendil | 5 min read

Next Post


Analyze data in motion with OCI GoldenGate Stream Analytics

Julien TESTUT | 3 min read