Data, process, and application integration convergence
By llaszews on Jul 19, 2011
Data integration is typically the domain of the organization and people that manage the databases. The products used to integrate data in theses organizations are a combination of custom data transfer and load scripts, ETL and ELT products, and in some cases database stored procedures. Application integration, on the other hand, is often done by the application developers using custom written applications, message brokers or an enterprise service bus. Process integration is often done by developers but can also be done by business analysts, or even business users. Process integration uses a workflow engine, BPEL product, or a proprietary orchestration engine like Microsoft BizTalk to orchestrate a set of processes into one business flow. Each of these technologies have traditionally offered solutions focused in their own domain: • Data Integration – Used to unload, move, transfer and load data from one database engine to another database engine from the same or different vendor. Also, used to load flat files received from outside or within the company, and extract flat files from the database. • Application Integration – Focused on applications communicating with each other to complete a business transaction, or exchanging data and messages between two or more applications. Many times used instead of data integration so that application business logic can be applied to the data before, after or during processing. • Process Integration – Process integration combines a set of standalone processes into one end-to-end business process. Process integration usually involves integrating with many different applications, data sources, or web services in order to fulfill on business flow such as a purchase order. What is happening is that each of these integration technologies is adding capabilities of the other two technologies: • Data Integration – Data integration products all have web services support. This has not always been the case. Web Services exposed in data integration products can be consumed by applications or business processes. This makes integrating data integration products with applications or business processes very easy. ELT and ELT products have always had robust workflow engines that are very similar in functionality to a process integration product such as a BPEL Process Manager. • Application Integration – Application integration products now all have data adapters for the major relational databases, content management repositories, XML files and flat files. This opens up the option of doing both application and data integration at the application tier. • Process Integration – Process integration products, like Oracle BPEL Process Manager, have a number of adapters from message, to flat file and relational databases. BPEL Process Managers were built from the ground up to support web services so they obviously can communicate with any application, data, or another process that is exposed as a web service. Each of these products will probably continue to be sold separately by IT vendors. However, the focus on orchestrating business processing using a BPEL Process Manager, exposing applications, process and data as web services will cause these separate products to act and look more a like one product than separate products. The adoption of SCA is expediting the convergence of data, application and process integration.