Wednesday Jan 30, 2013

Capture Application Events using Oracle Goldengate

In my past I have worked on SOA, BPM and Event Processing systems. I was lucky enough to have an opportunity to be part of teams building large SOA applications, Application Integrations and Application Enhancements.


One of the KEY problems that we faced when building Event processing or SOA systems is the problem of capturing the application events WITHOUT -
1.    Re-engineering/modifying the existing application
2.    Adding Database Triggers
3.    Build/utilize application provided hooks


All of the above methods are either intrusive or have a processing overhead on the Source Application processing. Thus most customers have to evaluate which of the above three methods is suitable for their environment and decide which method to select. As these methods require changes to the application architecture (code changes) or have performance implications some customers have second thoughts on going ahead with the project though they can see the value that they can derive from SOA, BPM and Event Processing systems. The customers who went ahead with these projects also had to factor additional cost in terms of services to re-engineer the application or hardware to handle the additional load.


Oracle Goldengate provides non-intrusive log-based data capture and extraction mechanism. A very little talked -about feature of Oracle Goldengate is its Java/JMS integration. Using Oracle Goldengate it is possible to capture transactions happening in the Application and write the transaction/event to a JMS queue.


With the knowledge of the existing application architecture and Data Model, it is possible to configure the Oracle Goldengate extracts to read transactions on from the specific tables impacted during an application event.

The diagram depicts how Goldengate can be used to capture Application Events non-intrusively.

Goldengate with JMS


As can be seen from the above representation


•    GG Processes run outside the Database
•    GG processes read the data from the Log files without hitting the source database
•    GG Pump process pushes the data in an online fashion to the Data Integration Server with minimal lag
•    GG Replicat processes read the data from the Trail files and post it to JMS
•    From JMS it can be fed into multiple different destinations as listed

This paves the way for non-invasive event capture from existing applications. Now it is possible to capture Application events without re-engineering the application or adding any significant load on the application.


This enables the extension of Existing applications for-


1.    Capturing events from the existing application and process those using Event processing engines like Oracle CEP.
2.    Captur
ing events from the existing application and trigger BPEL or BPM workflows using products like Oracle BPEL or BPM
3.    Captur
ing events from the existing application and build real-time dashboards using tools like Oracle BAM
4.    Extending the application functionality by capturing the events and performing Java actions (Integrating with Java Apps)
5.    Captured events can also be put into a Coherence Grid for any kind of in-memory grid processing of the events


Thus Oracle Goldengate can be used to capture application events for an SOA WITHOUT

1.    Re-engineering/modifying the existing application

2.    Adding Database Triggers

3.    Build/utilize application provided hooks


Goldengate enables non-invasive capture of the application events and deliver them to an SOA.


References and further reading-


1. A detailed step-by-step guide to capture events and writing to JMS Queues is available on the Oracle-By-Example site here.


2. Data Sheet for the Goldengate Application Adapters (which includes JMS support) is available here


About

This Blog discusses new ways of using Oracle Technology to achieve Integration, Harmonization and Orchestration for New and Existing Applications

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today