Friday May 20, 2016

Using Oracle Data Integrator (ODI) for Loading to Oracle Cloud’s Human Capital Management (HCM)

The A-Team is on a roll…

To load data into Oracle Cloud’s Human Capital Management (HCM) with Oracle Data Integrator (ODI) – check out the following blog:

Oracle Data Integrator (ODI) for HCM-Cloud: a Knowledge Module to Generate HCM Import Files

HCM is unique in that it uses a dedicated file format that contains both metadata and data. As far as the data is concerned, the complete hierarchy of parent and children records must be respected for the file content to be valid. This article explores a new Integration Knowledge Module (KM). This KM allows us to leverage ODI to prepare the data and generate the import file. Then, traditional Web Services connections can be leveraged to load into HCM.

Monday Jan 25, 2016

Oracle Metadata Management (OMM) 12.2.1 is now available!

Metadata Management is essential to solving a wide variety of critical business and technical challenges which include how report figures are calculated, understanding the impact of changes to data upstream, providing reports in a business friendly way in the browser and providing reporting capabilities on the entire metadata of an enterprise for analysis and improvement. Oracle Metadata Management is built to solve all these pressing needs for customers in a lightweight browser-based interface.

With Oracle Metadata Management 12.2.1, you will find many new features and enhancements. To list just a few:

Oracle Metadata Management is now based upon 100% Java delivery and installation allowing support for Windows as well as variation Linux/Unix deployments. This provides all UI features and any bridges which are executable in a Java and Linux/Unix based environment, specifically 100% java based bridges including JDBC database bridges (Oracle, Teradata, DB2, SQL Server, etc.), big data bridges (Hadoop Hive, HCatalog), and other popular bridges such as CA ERwin xml, Informatica PowerCenter xml, Tableau BI, etc.

Oracle Metadata Management configuration architecture visualization is now based upon an HTML 5 based implementation which includes better scalability, performance and overall layout quality, and supports iPad and other tablets.

Oracle Metadata Management provides two primary user interfaces for access, analysis, management and administration. The Metadata Manager UI provides a more technical interface for those who are managers of the metadata or administrators of the repository. The Metadata Explorer UI is designed for optimized search, analysis and reporting, as well as more control over the context (configuration version) presented to given users.

Oracle Metadata Management is now offering detailed DI data flow lineage analysis on demand only (in real time), instead of pre-calculating. This enhancement means that harvesting times are reduced by about 80%!

Oracle Metadata Management has a great number of new or enhanced bridges covering RDBMS, Big Data, Data Modeling, Data Integration, and Business Intelligence.

More details on the bridges and all the New Features for this release can be found here.

You can download Oracle Metadata Management 12.2. 1 here!

Friday Dec 18, 2015

Oracle Management Pack for Oracle Data Integrator for Enterprise Manager 13.1 is out! Make the most of the value of your Oracle Data Integrator investments!

We’ve mentioned before that to help you make the most of Oracle Data Integrator, and to deliver a superior ownership experience in an effort to minimize systems management costs, Oracle’s Management Pack for Oracle Data Integrator is a great addition! As a quick refresher: this product leverages Oracle Enterprise Manager Cloud Control's advanced management capabilities to provide an integrated and top-down solution for your Oracle Data Integrator environments. Management Pack for Oracle Data Integrator supports both 11g (11.1.1.7.0 and higher) and 12c versions of Oracle Data Integrator (12.1.2, 12.1.3 and 12.2.1).

Management Pack for Oracle Data Integrator provides a consolidated view of your entire Oracle Data Integrator infrastructure. This enables users to monitor and manage all their components centrally from Oracle Enterprise Manager Cloud Control.


In review, important capabilities include:

Performance Monitoring and Management

Management Pack for Oracle Data Integrator streamlines the monitoring of the health, performance, and availability of each and all components of an Oracle Data Integrator environment – this includes Master and Work Repositories, Standalone and JEE agents, as well as source and target Data Servers.

Configuration Management

To ensure that the configurations of all critical Oracle Data Integrator components in production environments are consistent with staging or test environments, Management Pack for Oracle Data Integrator provides Configuration Snapshots to save working configurations into the Management Repository or into an external XML file. These snapshots can then be compared with the active configuration in staging or test environments, or against any historical snapshot. Configuration Comparison is the next logical step which thus allows you to ensure the consistency of configurations in your application environment. Going one step further, Configuration History keeps track of all configuration changes, which can help in diagnosing performance problems that may be related to configuration changes along the way.

Service Level Management

Management Pack for Oracle Data Integrator allows for a proactive method to monitor Oracle Data Integrator environments at all levels. Many out of the box metrics are automatically collected for each Oracle Data Integrator component (repositories, agents, sources, targets…) which then provides users the ability to set up alerts based on critical or warning thresholds. Notifications can be provided to users in various ways through emails or Simple Network Management Protocol (SNMP) options.

Historical Analysis and Reporting

While real-time metrics certainly come in handy, there is also something to be said for historical collection of data! Management Pack for Oracle Data Integrator stores collected metrics and configurations data in a central repository –allowing users to thus analyze the metrics in various incremental views that include; last 24 hours, last 7 days, last 31 days. In addition, customizable service and system dashboard functionality allows users to create reports on various services and systems for service level availability, usage, performance, and business indicators – all in an effort to facilitate strategic trend analysis and reporting.

In a nutshell, Oracle Management Pack for Oracle Data Integrator benefits include: reducing the costs associated with monitoring the overall Oracle Data Integrator infrastructure and providing visibility into complex Oracle Data Integrator deployments across the enterprise, minimizing troubleshooting and performance tuning efforts as well as improving the overall ability to efficiently monitor Service Level Agreement compliance using powerful alerting capabilities. (For additional detail and to review another blog on the subject, click here.)

Oracle Management Pack for Oracle Data Integrator is licensed separately. From an installation perspective it is packaged in the Fusion Middleware plug-ins for Enterprise Manager (version 12.1.0.4 and higher) and comes with the base Enterprise Manager 12c or 13c download. There is no separate installation file to download. More information can also be found here.

Try it out! Getting Started with Oracle Fusion Middleware Management Plug-in

Friday Dec 11, 2015

Oracle Data Integrator 12.2.1 - Release Management with Deployment Archives

In earlier articles on ODI Lifecycle Management features we covered Managing Versions in Apache Subversion and Parallel Development using Subversion Branches. Now let’s explore the Release Management aspects addressed in ODI 12.2.1.

Deployment Archive

The release management use cases are addressed in ODI 12.2.1 through the newly introduced object called Deployment Archive. It is an archive file containing a set of ODI objects along an XML file containing archive details. It is the vehicle to promote objects from a development environment to the testing or production environment as depicted in below diagram. In a typical release process a Deployment Archive is first deployed in testing environment where the validations can be performed and after testing is done it is applied to a production environment.


Deployment Archive Types

There are four types of Deployment Archives

  1. Initial Deployment Archive – It is used to initialize a fresh ODI repository. It is created with all the objects in your development project.
  2. Patch Deployment Archive – As the name suggest it is used to apply patch containing a bug fix or enhancement to an ODI repository that is already initialized with Initial Deployment Archive. It usually contains only the object having the bug fix or enhancement and all its dependencies.
  3. Rollback Deployment Archive – A Rollback Archive facilitates restoring a pre-patch state of deployment environment. It is created when a Patch Deployment Archive is applied to an ODI Repository and contains the artifacts from the repository that are being updated by the patch.
  4. Execution Deployment Archive – It holds only the runtime executable objects and gets created along with Initial or Patch Deployment Archive. It is used to promote artifacts to ODI execution repository, which holds only executable objects.

Creating Deployment Archive

All Deployment archive related operations can be invoked from the ODI Studio menu. There are three options available for creating a Deployment Archive, which differs in the way the objects to be included in the Archive are selected.


 Each of these options allows you to
  • Select the type of Deployment Archive to create (if applicable)
  • Provide Archive Name
  • Specify the file location where the archive is created
  • Specify an Export key which is used to encrypt sensitive data in the exported object XML
  • Provide an option to exclude physical topology containing connectivity details.

Create from a VCS Label (Tag)

The objects from a Subversion Tag are included in the Deployment Archive.

Create from Repository > Full Repository

All the objects from the entire repository are included in the Deployment Archive. Since all the objects are included here, with this option you can create only the Initial Deployment Archive.


Create from Repository >Selected Objects

You can selectively add the objects you wish to include in the Deployment Archive. All dependencies of an object is also automatically included along with the object so that the Deployment Archive always contains a consistent set of objects that can be applied in any environment.

Viewing Deployment Archives

You can list and view the details of Deployment Archives from ODI Studio by specifying the folder location where all the Deployment Archives are created.


You can double click on any entry to the list of objects present under that archive.


Applying an Deployment Archive

An empty repository can be initialized with the Initial Deployment Archive.


If the repository is already initialized then you can apply a Patch deployment Archive to it.


While applying patch archive you can also specify the file location where the corresponding Rollback Deployment Archive should be created. A Rollback Archive is used to restore the pre-patch state of the repository when the Patch application result in some problems. 

As shown below, a typical Production environment is first initialized with the released artifacts from a Initial Deployment Archive and then at some later point in time the Patch Deployment Archive is applied for some bug fix or enhancement.


Auditing Deployment Archives

ODI maintains the history of all the Deployment Archives applied to a repository and it can be easily view in ODI Studio.


The Inventory table for the applied patches has a number of columns as shown above providing important information.
  • It provides details about the Deployment Archive- its type, how it was created, who created it and when it was created.
  • It shows in what order a Deployment Archive was applied, by whom and when.
  • It provides the statistics - how many object got added, updated or deleted as part of it.
  • It shows if the Deployment Archive can be rolled back and the path where the corresponding Rollback Deployment Archive was created.

Rolling Back a Patch Deployment Archive

The patches for which a Rollback Deployment Archive was created can be rolled back from studio.


Note that not all Deployment Archives can be rolled back. In order to be rolled back it should satisfy following conditions.
  1. It should be a Patch Deployment Archive
  2. The corresponding Rollback Archive should have been created during patch application
  3. The corresponding Rollback Archive should be present in the specific file location
  4. The patches can be rolled back in the sequence they are applied. It means that the patches can be rolled back in Last In First Out (LIFO) order only. 

Programming Interface for Automation

There is Software Development Kit (SDK) Application Programming Interface (API) available for all the Release management activities described in this article. The deployment archive management related classes are available in oracle.odi.core.service.deployment package. Using these APIs you can write programs to automate your release process.

Conclusion

ODI 12.2.1 has come out with a number of features addressing different Lifecycle Management needs. It provides seamless integration with Apache Subversion to manage ODI objects versions in external version control system. It provides Branch and Tag management capabilities to facilitate parallel development between functionally or geographically distributed teams. It provides comprehensive release management capabilities to create Deployment Archives with artifacts from development environment which can then be tested in isolation, applied to production environment and can be rolled back if required. With all these capabilities users can easily manage an objects lifecycle in ODI 12.2.1, which is now generally available to download from OTN page.

Friday Oct 23, 2015

Announcing Oracle Data Integrator (ODI) 12.2.1 – Focus on Big Data and Lifecycle Management Innovations

Oracle Data Integrator 12.2.1 makes its debut today, with new and enhanced functionality in many aspects of its core capabilities. Take a look here for the latest details – and read on for some highlights:

Big Data

Back in April we proudly announced the availability of Oracle Data Integrator for Big Data. The release brought together a series of advanced Big Data updates and features that Oracle Data Integration is rolling out for customers to help take their Hadoop projects to the next level with support for Apache Spark, Apache Pig, and orchestration using Oozie. Today, we continue to enhance this functionality by allowing users to choose between Task and Session execution modes for Oozie workflow generation.