Solving EAI with SOA Suite

By Tejas Joshi :Those of us who have been around for a while in IT, would know that we tend to solve same problems with new ways of thinking. I remember hype around EAI and working on number of projects 7 - 8 years ago in solving integration issues for very complex systems.
Some of the challenges that were dealt with were

  1. Standardize the integration interfaces across business systems
  2. allow for protocol switching
  3. allow for data mapping between source and target systems
  4. Transactional integrity with compensation logic
EAI resulted in very complex solutions. These solutions were not only brittle but also costly to manage as they were based on proprietary technology. But it did the job for providing businesses with integrated systems.
Some of the pattern in EAI were
  1. Hub-and-Spoke: example of hub and spoke concept refers to a parent or holding company that uses one business software system (the hub), which is integrated with the systems used business units within its fod.
  2. Bus: Example of Message Bus is a combination of a common data model, a common command set, and a messaging infrastructure to allow different systems to communicate through a shared set of interfaces.

But we ended up with following disadvantages
  • High initial development costs, especially for small and mid-sized businesses (SMBs)
  • Require a fair amount of up front business design, which many managers are not able to envision or not willing to invest in.
  • Most EAI projects usually start off as point-to-point efforts, very soon becoming unmanageable as the number of applications increase
  • Costly and time-consuming to maintain tightly coupled point-to-point integrations
  • Service premiums for evolving proprietary integration broker implementations
  • Static data integrations with proprietary interfaces are difficult to extend to the web

With advent of SOA, a new way of thinking evolved which allowed us to look at integration as not just IT challenge but a business challenge. This forced us in thinking for business benefits, allowed us to priorities integration across systems that were business critical versus everything. We looked at building business functionality through reuse or service enablement rather than point integration between systems.
Recently, I was involved in migration project of an old EAI based Hub-n-Spoke systems to SOA Suite. It became quite apparent that even though our mandate was a clear migration from proprietary EAI system to SOA Suite, we could simplify a very complex solution using principles of SOA and SOA based tooling.
We had to migrate over 200 integration points across 15 systems with SOA Suite. While the requirements were captured as system behaviour, during analysis phase we realised that there were distinct integration patterns emerging around two principles
a) protocol used and message type
b) System data structure mapping from source to target
We were able to map a single integration pattern with 4 high level components
arch1.gif

The pattern focused on four reusable services that formed the core of the solution.
  • Inbound Reader Service: This service would accept inbound messages using required messaging protocol and type. Oracle Service bus is best placed to implement such services, which could be configure to read inputs across multitude of protocols in forms of proxy services.
  • Structural Validator Service: This service would validate the messages to be structurally correct. This will ensure that the message processor service will only process valid messages. Example of such validation could be schema validation, file structure validation etc.
  • Message processor Service: This service would be detailed business logic based orchestration to run business validation and transformation. SOA Suite based SCA could be used to implement this in BPEL. To make this process generic BPEL XPATH functions could be used to dynamic transformation.
  • Message Publisher: This service would use dynamic content based routing to publish message to target system. This could be either implemented in SCA (mediator) or OSB.

The proposed solution based on SOA Suite can replace EAI for fraction of the original project cost and still maintain the essence of Hub-n-Spoke pattern
sol1.gif

Comments:

Hey Tejas, Nice article, good to see you on the blogosphere! Shaukat

Posted by Shaukat Desai on October 19, 2010 at 04:17 AM BST #

Hi Tejas, We are considering such a migration project from EAI to SOA suite. The question is : If there are multiple applications communicating and have fairly similar requirements 1. Different Flat/XML file structures 2. Different schedules for sending/Receving files. Which introduces message queueing. 3. Diffterent protocols for sending/receiving files Although applications are faily communicating the same way, I think this approach will result in many Proxy Services, JMS queues and BPEL processes essentially doing similar things of transformation/routing. If there are 100 applications talking to each other should there be 100 OSB/BPEL projects? And how to handle Message Requeueing,monitoring etc. If you can explain how did you handle it. it will be helpful. Thanks, Ambadas.

Posted by Ambadas on October 28, 2010 at 01:50 PM BST #

Ambadas, You can rely on dynamic routing and dynamic transformation to process different types of messages and handling which message goes where. XQuery is brilliant tool for dynamic transformation in OSB and SCA based Business rules based routing to map messages to back end system. You would end up with per channel (i.e. File, FTP, Email) implementation. If you are interested I would describe one of the patterns in more detail with an inbound and out bound example in my next blog. The point of the SOA approach is to avoid point to point mapping, the problem you describe is exactly what we avoided.

Posted by tejas.joshi on October 28, 2010 at 02:47 PM BST #

Hi Tejas, Thanks for the quick reply. If you can explain the above pattern with an example it would be helpful. In such a solution, right now I'm not able to imagine, how 2 new applications with following requirements would be added. 1. Application A and B want to send flat files to each other. App A sends files in 4 hour Batch and App B picks up messages only in the nightly single batch and replies back. 2. Say app A uses JMS to post messages and App B expects files in FTP. How this requirement will be satisfied with the existing design. I'll wait for your next blog :) Thanks, Ambadas.

Posted by Ambadas on October 29, 2010 at 06:58 AM BST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Tech tweets are technical ideas and solutions that I might have recently come across while working on Oracle Fusion Middleware.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today