Tuesday Jun 28, 2011

Join our webcast: Discover What’s New in Oracle Data Integrator and Oracle GoldenGate

Data integration team has organized a series of webcasts for this summer. We are kicking it off this Thursday June 30th at 10am PT with a product update webcast: Discover What’s New in Oracle Data Integrator and Oracle GoldenGate. In this webcast you will hear from product management about the new patch updates to both GoldenGate 11g R1 and ODI 11gR1.

Jeff Pollock, Sr. Director of Product Management for ODI will talk about the new features in Oracle Data Integrator, including the data lineage integration with OBI EE, enhanced web services to support flexible architectures as well as capabilities for efficient object execution such as Load Plans. Jeff will discuss support for complex files and performance enhancements.

Chris McAllister, Sr. Director of Product Management for Oracle GoldenGate will cover the new features of Oracle GoldenGate such as increased data security by supporting Oracle Database Advanced Security option, deeper integration with Oracle Database, and the expanded list of heterogeneous databases GoldenGate supports . Chris will also talk about the new Oracle GoldenGate 11gR1 release for HP NonStop platform and will provide information on our strategic direction for product development.

Join us this Thursday at 10am PT/ 1pm ET to hear directly from Data Integration Product Management . You can register here for the June 30th webcast as well as for the upcoming ones in our summer webcast series.

Thursday May 26, 2011

Oracle releases Oracle GoldenGate

The Oracle Data Integration team recently released a bundled set of patch updates for Oracle GoldenGate 11gR1 which includes many new features and capabilities. This release is certainly exciting as it demonstrates Oracle’s unique strategy to continue to integrate its best of breed solutions forming a pervasive data integration platform.

Below is only a highlight of the features included in this patch set and I encourage you to view more details provided in the links below:

  • Improved Monitoring
    • Manager Infrastructure improvements provide better client/server network communication, better port use reporting, newly designed modular command processor, and improved process management
    • New Manager Agent includes embedded JVM to support new Oracle GoldenGate Monitor to retrieve monitoring points from shared memory
  • Increased Security
    • Oracle GoldenGate furthers its data replication capabilities by supporting replication of data protected by Oracle Advanced Security data encryption options. This includes data protected by either column-level encryption using Transparent Data Encryption as well entire contents of data files using Tablespace Encryption
  • Expanded Capture and Delivery support for Heterogeneous Platforms
    • SQL Server 2008 R2
    • Sybase ASE 15.5
    • DB2 v10 for z/OS
  • Tighter Integration with Oracle Products
    • Oracle GoldenGate Capture module is integrated tighter with Oracle Recovery Manager (RMAN) by providing recovery checkpoint information within system tables which protects that archive logs are available when required. Also improves archive log management reducing space requirements when archives log are not longer required.
    • Oracle GoldenGate expands data type coverage to include timestamps with region ID
    • New schema-level supplemental logging provides automatic supplemental logging for newly created and altered tables

To view a complete list of all the new features, download and read the Oracle GoldenGate release notes.

For more information about installing and configuring Oracle GoldenGate to take advantage of these features, or just for help with upgrading your current installation to the newest release, visit the user library.

Wednesday May 25, 2011

Oracle Data Integration 11g EStudy

An eStudy called Oracle Data Integration 11g, New Features is available to everyone. In less that one hour, you can learn all you need to know about 11g. 

Oracle Data Integration 11g: New Features

This tutorial introduces you to the new and enhanced features of Oracle Data Integrator 11g (, including an understanding of the business reasons and technical benefits of these features. The course is NOT a general introduction to all aspects of the Oracle Data Integrator product. You should have thorough familiarity, preferable hands-on, with the previous 10g version of Oracle Data integrator. After completing this course, you should be able to: 

- Identify changes in the Oracle Data Integrator 11g architecture and components 

- Describe components of the new Fusion (JDeveloper) user interface 

- Identify the new and enhanced usability and productivity features 

- Describe the enhancements to Knowledge Modules 

- Describe the changes to ODI agents 

- Explain the business benefits of each new or enhanced features

Click here to view the tutorial, and enjoy ! 

Saturday May 21, 2011

Extending the Interface Accelerator

Carrying on the automate, automate theme, here’s an illustration of some really useful capabilities using the ODI 11g SDK which make it more powerful, carrying on from the post on the simple accelerator that I posted here. In this posting I had a simple control file that mapped some table to some other table that were defined in in the control file. The auto mapping as-in the ODI UI itself matches on exact names of columns.

What if we want to do something a little different and automate? For example say some standard suffixes or prefixes have been added to the column names what can we do? With the previous posting no columns will be mapped. This was because I used the auto mapping provided out of the box (see the following line in the interfaceAccelerator.java file from the post mentioned;

interactiveHelper.performAction(new InterfaceActionSetTargetDataStore(targetDatastore, new MappingMatchPolicyLazy(), new AutoMappingComputerLazy(), new AutoMappingComputerLazy(), new TargetKeyChooserPrimaryKey()));

Here is where we can refine this by specializing the auto mapping class that ODI provides, I will create one called MyRegexpAutoMappingComputer.java this class will refine the method performAutoMapping which will construct the mapping for a target column – it can even call the super classes method and reuse the default exact name matching if needed.

interactiveHelper.performAction( new InterfaceActionSetTargetDataStore(targetDatastore, new MappingMatchPolicyLazy(), new MyRegexpAutoMappingComputer( MyRegexpAutoMappingComputer.MatchTypes.TGTENDSWITH), new MyRegexpAutoMappingComputer( MyRegexpAutoMappingComputer.MatchTypes.TGTENDSWITH), new TargetKeyChooserPrimaryKey()));

You can see in the above call I now use a new auto mapping class and pass in the TGTENDSWITH enumeration to do matching (so a source call named EMPNO will match with a target name of STG_EMPNO). The new files are available here;

The class could be enhanced to support more complex stuff, hopefully you see how this works and an insight into how you can extend the SDK provided and add even more value.

Monday Apr 25, 2011

Accelerating development via the Oracle Data Integrator SDK

A nice post and example of ODI 11g SDK use by David Allan can be seen on this blog.

David writes:


Often using ANY tool there are scenarios where there is a lot of grunt work, imagine Microsoft tools like Excel without VB and macros to accelerate and customize those boring repetitive tasks. Data integration and ETL design is exactly the same, the tool needs to expose an SDK to a base platform that you can use to make your life easier. Something to automate the grunt work that is common and very repetitive. The ODI 11g SDK let's you script these kind of repetitive tasks. As an aside the ODI common format designer (see this post here) has a way for migrating like named objects, however using the SDK let's you control much much more.

To illustrate I have created a simple interface construction accelerator that you can download (interfaceAccelerator.java), the accelerator generates ODI interfaces from a control file that defines the interface name, the source and the target - simple and a nice example for demo purposes. If you look at the java code, it is very basic (no pun intended).  It literally is a dozen lines of code. The image below illustrates the java program interfaceAccelerator using the ODI 11g SDK to take as inputs the configuration of the connection details and a control file specify the source to target datastore mappings.


The code when called has a bunch of command line parameters shown below and the standard input stream is the interface control file, so the command line looks like;

java -classpath <cp> interfaceAccelerator <url> <driver> <schema> <pwd> <workrep> <odiuser> <odiuserpwd> <project> <folder> < <control_file>

the control file provided in the standard input stream needs to be a comma delimited file with the following structure

    • interface_name,source_model,source_table,target_model,target_table
    • ...

for example a sample command line using an Oracle repository could be

java -classpath <cp> interfaceAccelerator jdbc:oracle:thin:@localhost:1521:ora112 oracle.jdbc.OracleDriver ODI_MASTER mypwd WORKREP1 SUPERVISOR myodipwd STARTERS SDK < icontrol.csv


the interfaces will be created in the folder SDK and the project code is STARTERS. The icontrol.csv file used above was (remember the format is interface_name,source_model,source_table,target_model,target_table, this is just what I happened to use in this simple demo program);


You can created as many interfaces from this driver control file as you desire, the interface generated will map from the source table to the target table and use ODI's auto mapping to perform column level mapping of the source to target table, it will also create default source sets and use the default KM assignment. So you get a pretty useful set of stuff as a basis here.

The interfaces generated whilst executing this accelerator look like the following, the table to table map with all of the like-named columns mapped, the physical flow configured with defaults KMs!


You can take this code and customize to make it fit your needs or send in comments on how to do things. In summary if you are finding you desire ways of tuning your work to make using ODI even more productive, then you should look into the ODI 11g SDK and see if you can automate, automate, automate.



Learn the latest trends, use cases, product updates, and customer success examples for Oracle's data integration products-- including Oracle Data Integrator, Oracle GoldenGate and Oracle Enterprise Data Quality


« July 2016