Written by Ayush Ganeriwal, Oracle Data Integrator Product Management
In the previous blog post we looked at the Fusion Applications
end-to-end bulk data integration use cases. Now let’s take a closer look at the
Bulk Import process that transforms and moves data from Interface tables to
internal tables. For this use case ODI is bundled along with Fusion Application
and get configured transparently by the Fusion Application provisioning
process. The entire process is automated and controlled through the Fusion
Application User Interface. It also seeds the ODI repository with the Fusion
Application specific Models, Interfaces, Procedures and Packages which are then
dynamically modified through ODI SDK for any Fusion Application customizations.
Fusion Application Bulk Import Process
The above diagram shows the Bulk import process in Fusion
Application where ODI is used for data transformation. Here the Interface
tables are the source tables which were populated by other processes before the
kicking off the Bulk Import process. The Fusion Application internal tables are
the target for these integrations where the data needs to be loaded. These
internal tables are directly used for Fusion Application functionalities
therefore a number of data validations are applied to load only the good
quality data into the internal tables. The data validation errors are monitored
and corrected through Fusion Application User Interface. The metadata of Fusion
Application tables is not fixed and gets modified as the Application is
customized for customer’s requirement. Any change in such source or target
tables would require corresponding adjustments in ODI artifacts too and is
taken care of by the AppComposer which uses ODI SDK to make such changes in ODI
artifacts. If auditing is enabled then any change in the internal table data or
the changes in ODI artifacts are recorded in centralized auditing table.
Packaged ODI Artifacts
are a large number of ODI models, interfaces and packages seeded in the default
ODI repository used for Bulk Import. These ODI artifacts are built based upon
the base metadata of Fusion Application schema.
part of the customization, Fusion Application entities are added or modified as
per the customer’s requirement. Such customizations result in changes in the
underlying Fusion Application’s internal tables and interface tables, and
require the ODI artifacts to be updated accordingly. The Fusion Application
development team as built the extensibility framework to update ODI artifacts
dynamically along with any change in Fusion Application schema. It leverages
the ODI-SDK for performing any changes in the ODI repository. The dynamic
generation of ODI artifacts is automatically kicked off as part of Patching and
Upgrades process. Fusion Application AppComposer User Interface also supports
explicitly triggering this process so that administrators can regenerate ODI
artifacts whenever they make any customizations.
Validation Error Reporting
validation errors are populated in intermediate tables and are exposed through
BI Publisher so that admin users can correct and recycle these error records.
Fusion Application auditing framework keeps track of the changes performed by
each of the users and at what time. There are two levels of auditing captured
in Fusion Application audit table for Bulk Import use case. First, metadata
changes in ODI artifacts through ODI SDK during customizations. Second, the
transactional data changes in the Fusion Application table data as part of ODI
interfaces execution. For these purposes the ODI team has exposed some
substitution APIs that are used by Fusion Application development team to
customize ODI KMs to perform such auditing during the actual data movement.
Provisioning and Upgrade
provisioning process takes care of install and configuring ODI for the Fusion
takes care of automatically creating ODI repository schemas, configuring
topology, setting up ODI agents, setup configurations for ODI –ESS bridge,
seeding packaged ODI artifacts, apply modifications to seeded artifacts and
create internal users in IDM for external authentication. There is a separate
process to apply patches or upgrade the environment to the newer release. Such
patching or upgrade processes not only take care of importing newer ODI
artifacts but also kick off a CRM extensibility process that modifies ODI
artifacts as per the Fusion Application customizations.
is a dedicated IDM configured with each Fusion Application instance and all Fusion
Application components are expected to have their users authenticated through
this centralized IDM. For Bulk Import use case ODI is configured with external
authentication and there are internal users created in IDM that are used for
communication with ODI agent and kicking off ODI jobs.
Enterprise Scheduler Service (ESS) - ODI
ODI scenarios are kicked off through ODI-ESS bridges. It is a separate library
build for ODI-ESS integration and gets deployed along with Enterprise Scheduler
Service (ESS) in Fusion Application environment. It supports both synchronous
and asynchronous modes of invocation for ODI jobs. In the asynchronous mode the
session status is updated to callbacks to the ESS services. There is a topology
editor provided to manage the ESS callback service connectivity exclusively for
Fusion Application use cases.
Use of ESS-ODI Bridge is restricted to Fusion Application use case only at the
ODI agent is deployed on Weblogic cluster in the Fusion Application environment
to take advantage of ODI high availability capabilities. By default there is
only one managed server in the Weblogic cluster created for ODI but as the load
increases more managed servers can be added to the cluster to distribute
execution of ODI sessions among ODI agent instances in the cluster.
Stay tuned for the last post on this topic coming soon. This was part two in a series of three posts. The initial post can be found here.