Oracle GoldenGate brings an innovative approach for replicating full-featured business objects in an event-based, pub/sub architecture. This innovation combines two new Oracle technology features: Oracle Database JSON Relational Duality Views and Oracle GoldenGate Data Streams.
Managing and storing data efficiently for application development has faced significant challenges since the first data API had been created. Typical shortcomings in application development (app dev) patterns put more onus on the developers to constantly evolve the data access API to suit the data consumers’ requirements. In this blog, we will dive into these challenges to show how new ways of thinking, combined with a modern data fabric infrastructure, plus two new revolutionary features introduced by Oracle Database 23ai (JSON Relational Duality Views) and Oracle GoldenGate 23ai (Data Streams) all help to address those challenges.
Familiar challenges with app dev:
Most application developers are familiar with these typical challenges:
- Impedance mismatch: Application object structures don’t match optimized relational/normalized schemas, causing complexity in the application code to keep data in sync between the data structures.
- EAV anti-pattern: Developers tend to use an Entity-Attribute-Value (EAV), like a Key-Value-Pair (KVP), to store data in rapidly evolving applications, but this becomes unwieldy at larger scales. Developers are now repeating such history, by using JSON docs as EAV/KVP.
- Atomic updates to event brokers: How to atomically update the database and send those events to/through a message broker (while maintaining consistency and durability) for microservices apps.
- Exposing the physical model with CDC: Databases and Change Data Capture (CDC) tools can send guaranteed transaction logs to consumers, but usually only send row-level views of the data (not app objects).
Many technical solutions (including object relational mapping [ORM], transaction outbox patterns, transaction log tailing patterns, etc.) have tried to solve each of these challenges, over the years. However, each has fallen short, while also/potentially introducing other problems, leading to more headaches for data API developers.
Introducing an elegant solution for event-driven business objects:
Oracle recently introduced two landmark innovations within the Oracle Database 23ai and Oracle GoldenGate 23ai.
Oracle Database 23ai JSON Relational Duality Views enables developers to easily consume relational data as JSON document models. This combines the advantages of JSON documents with those of the relational model, while avoiding the limitations of each. Oracle JSON relational duality view is essentially a SQL view that presents JSON data in a structured, relational format.
Replicating Business Objects with Oracle JSON Relational Duality and GoldenGate Data Streams prioritizes data products and becomes the single source of truth for applications, transaction fabrics, data/event meshes, and data fabrics by ensuring the uniqueness of the data objects.
Replicating Business Objects with Oracle JSON Relational Duality and GoldenGate Data Streams solves the above-mentioned challenges by bringing together the best of relational, document, change data capture and AsyncAPI worlds/capabilities:
- Overcoming impedance mismatch: In JSON Duality Views, data is stored only once, underneath the normalized tables (preserving atomicity), while application objects work with JSON data objects. This way, business objects manage data products in Oracle JSON Duality Views in forms of JSON objects. GoldenGate captures JSON based data objects from JSON Duality Views and publishes events to AsyncAPI channels to make it available for subscribing data consumers. Integration between JSON Duality Views and GoldenGate addresses the impedance mismatch by guaranteeing exact once availability of the data products for both data producers and data consumers.
- Scalable Data Objects: JSON Duality Views provide efficient storage management and data consistency as your application can work natively with JSON docs or with SQL on tables, or with both at the same time. Duality Views can be declared over any number of tables, plus even different duality views can be defined on the same or overlapping set of relational tables. This allows application developers to manage application objects in ways they are familiar and to create complex data products while using their usual drivers, frameworks, tools, and development methods, all while data is stored in a relational database.
- Atomic updates to event brokers: JSON Duality delivers faster JSON than document databases and also delivers better latency for OLTP by providing parallelism and optimization, as data management is done at the relational level with very high efficiency. GoldenGate Data Streams automates the event-based publication of business objects directly from the data tier, in real-time, as the COMMIT events happen, using standard patterns.
- ACID compliant data objects with CDC: In JSON Duality Views, Lock-Free Concurrency Control keeps data objects constantly in-sync, while preventing both inconsistencies and stale data. GoldenGate Data Streams publishes trusted JSON based data products with high throughput, reduced latency and guaranted exact once availability of the data product for all data consumers.
Learn more about Replicating Business Objects with Oracle JSON Relational Duality and Oracle GoldenGate Data Streams: