« December 2008 | Main | February 2009 »

January 2009 Archives

January 6, 2009

Breaking Radio Silence, and OWB and Oracle Exadata

Between the U.S. holidays and hard work on OWB 11.2 Beta, it was a slow month for blog-worthy activity on OWB. But rest assured, we're still toiling away on Warehouse Builder, and we will give you plenty to talk about quite soon.

Meanwhile, OWB will be featured in the upcoming HP Oracle Exadata Roadshow in New York City:


Oracle Exadata combines industry standard hardware from HP with smart software from Oracle that overcome the limitations of conventional storage and can improve data warehouse query performance by a factor of 10x or more. In addition to industry-leading performance and scalability Oracle Database 11g offers integrated analytics and embedded data quality on a reliable, low-cost grid infrastructure. This encompasses a broad set of database features and options that allow you to:

  • Manage integration and loading of quality data with Oracle Warehouse Builder

  • Improve performance, availability, and manageability by partitioning large tables

  • Consolidate enterprise wide reporting, analytics and data mining

Join us and learn how Oracle Exadata and Oracle Database 11g provide a comprehensive platform for addressing today's data warehousing and business intelligence requirements.

Tuesday, January 20, 2009
8:00 a.m. - 12:30 p.m.

Marriott Marquis at Times Square
1535 Broadway
New York, New York 10036
212.398.1900

Process Flow Parameters

Ever wondered how to pass values out of one process flow activity into another? The use of variables and activity parameter bindings is the key. In the example below process flows are used as the activities, the same applies for any activity with parameters.

In the image below, the process flow named MASTER has 2 child flows; CHILD and CHILD_1, an output parameter of the sub-flow CHILD is passed into the input parameter of another sub-flow CHILD_1. To do this, the output parameter must be stored in a local variable DATASTORE of the MASTER flow and this variable is then passed into the next activity as an input parameter.

owb_flow_param0.JPG

Another common query is how to actually add variables and parameters in the first place. This is done by selecting the Variables node or START1 node in the selected objects panel, and clicking on the green plus button, not exactly intuitive, but that's how its done;

owb_flow_param0.JPG

After the variable or parameter has been added, you can select it in the tree and change its properties in the Object Details panel. A couple of bits of info that its not obvious how they operate.

January 11, 2009

OWB Application Connector Interfaces Now Documented

Oracle Warehouse Builder Application Connectors provide simplified access to the underlying data from Oracle's ERP applications-- E-Business Suite, PeopleSoft, Siebel, and MDM applications.

The benefits of application connectors include:


  • The database objects exposed to the OWB user can be limited to those that correspond to the logical-level business objects managed by the application.

  • Those objects can be managed and used in OWB under meaningful business names instead of cryptic physical-level names, and grouped into business areas if the application provides such metadata.


The result is improved developer productivity on projects where the supported ERP applications are used as sources and/or targets for OWB in data integration, data warehousing and data quality management.

Oracle makes available application connectors for many Oracle ERP applications such as Oracle E-Business Suite. However, Oracle customers using applications other than those supported by official Oracle OWB connectors have similar requirements.

To help application vendors better serve these customers, we are opening the interfaces used to build OWB connectors for use by creators of Oracle-based applications. A new whitepaper provides all of the needed information to create your own connectors.

Continue reading "OWB Application Connector Interfaces Now Documented" »

January 16, 2009

Expert: Useful scripts from simple primitives

Here is a useful OMB scripting example to illustrate the basic mechanics of the OMB scripting grammar to create an expert. The expert has a couple of steps to get information from the user to load an object (file, table etc.) from one place to somewhere else, quite simple...and the expert itself is pretty simple too! The expert can be found on the utility exchange here, and can be found in OMB form here (just need to source the OMB to define the expert).

For details of the OMB command set see the OWB Api and Scripting reference (here) the OMB commands are simply extensions to a tcl shell which is based on a java implementation so you can also integrate java code.

When the expert is run the user is prompted for;
1. the location of the source
2. the object to be moved (let's call it source)
3. the location of the target (an object named T_source will be created)

After this information is collected the new target is created and the mapping to load it generated.

Select Source

The Select Source task in an expert will display a dialog that lets the user choose the source type and the location for that source type (or create a new location). The task was added in the tcl script using the following command;

OMBALTER EXPERT 'DATA_MOVER' ADD SELECT_SOURCE TASK 'SELECT_SOURCE_TASK'

dm1.JPG

A new location can be defined and the path entered (see the file example here);

dm2.JPG

The select source task has a number of inputs and outputs, these are bound to variables as follows in the expert;

ParameterDirectionVariable
OBJECT_TYPEOutG_SRC_OBJ_TYPE
MODULE_NAMEOutG_SRC_MODULE
LOCATION_NAMEOutG_SRC_LOC
SOURCE_TYPEOutG_SRC_MODULE_TYPE
PARENT_CONTEXTInOMB_CURRENT_PROJECT

This parameter to variable binding is done by using the following grammar (for example);

OMBALTER EXPERT 'DATA_MOVER' \
MODIFY PARAMETER 'OBJECT_TYPE' OF TASK 'SELECT_SOURCE_TASK' BIND TO VARIABLE 'G_SRC_OBJ_TYPE' \
MODIFY PARAMETER 'MODULE_NAME' OF TASK 'SELECT_SOURCE_TASK' BIND TO VARIABLE 'G_SRC_MODULE' \
MODIFY PARAMETER 'LOCATION_NAME' OF TASK 'SELECT_SOURCE_TASK' BIND TO VARIABLE 'G_SRC_LOC' \
MODIFY PARAMETER 'SOURCE_TYPE' OF TASK 'SELECT_SOURCE_TASK' BIND TO VARIABLE 'G_SRC_MODULE_TYPE' \
MODIFY PARAMETER 'PARENT_CONTEXT' OF TASK 'SELECT_SOURCE_TASK' BIND TO VARIABLE 'OMB_CURRENT_PROJECT'

That's it for selecting the location of the source, next we have to define what to import (for a file location this will be the file within the directory, for a database, the table within the schema etc.).

Object to Import

The import dialog will be

The Source Import (to import metadata) task in an expert will display a dialog that lets the user choose the object to import, this could be a file, a database table etc. The task was added in the tcl script using the following command;

OMBALTER EXPERT 'DATA_MOVER' ADD SOURCE_IMPORT TASK 'SOURCE_IMPORT_TASK'

dm4.JPG

Now with the file selected, the source import task will launch the file sampler wizard to define the metadata for the file. Had the object been a database table, the tables and columns would be imported (since the metadata is known, unlike with files).

So with this example you will then define the flat file metadata;
dm5.JPG


This wizard is the standard OWB flat file sample wizard. The Source Import task has a number of inputs and outputs, these are bound to variables as follows in the expert;

ParameterDirectionVariable/Value
RETURN_VALUEOutG_IMPORT_RESULT
IMPORT_MODEInthe value MINIMAL_MODE
OBJECT_TYPEInG_SRC_OBJ_TYPE (an output in previous task)
PARENT_CONTEXTInG_SRC_MODULE (an output in previous task)

When this is complete the final information is requested, where is the data to be moved to, what is the target?

Location of the target

The Select Target task in an expert will display a dialog that lets the user choose the target type and the location for that target type (or create a new location). The task was added in the tcl script using the following command;

OMBALTER EXPERT 'DATA_MOVER' ADD SELECT_TARGET TASK 'SELECT_TARGET_TASK'

dm12.JPG

Just like the select source task, the select target will let you create a new location for the target type desired.

The select target task has a number of inputs and outputs, these are bound to variables as follows in the expert;

ParameterDirectionVariable
OBJECT_TYPEOutG_TGT_OBJ_TYPE
MODULE_NAMEOutG_TGT_MODULE
LOCATION_NAMEOutG_TGT_LOC
SOURCE_TYPEOutG_TGT_MODULE_TYPE
PARENT_CONTEXTInOMB_CURRENT_PROJECT

As you see from all of these tasks they rely on variables to cooperate, one task produces information that is consumed by another and variables are used for this and any custom code the task may have/need.

What was produced?

When the expert is run using a file as a source the expert produces the file definition, an external table based on this file, a target table where the data is moved and a mapping that moves the data (along with the locations for these items). All in a simple guided example! Here is a snapshot of what was produced after a simple execution;

dm15.JPG

The OMB task at the end produced the target table and the mapping;

dm16.JPG

A neat trick was used in scripting to derive the target table; an unbound table was added to the mapping, the source mapped to it getting all of the columns and types, then the table operator was used with an outbound reconcile/synchronize to create the table. This was achieved by using the OMBRECONCILE command in the OMB task, see the datamover.tcl script referenced earlier.


This is a nice example which illustrates using OMB to create an expert itself, for some background on experts see the paper here and send on any comments and questions.

January 19, 2009

OWB Integration with Salesforce.com Data: Sometimes The Old Ways are the Best Ways

A request that we get frequently enough to comment on is, "How do I use OWB to move data between Salesforce.com and my Oracle database?" Data warehouse customers in particular seem to be running into this one.

Usually, the person asking expects that some kind of web service call will be the solution of choice. Perhaps it would be, in some arcane cases. There are also ODBC drivers for Salesforce.com, which OWB could connect to with similar results to other ODBC data sources. (Of course, Oracle support won't help with those third-party components.)


However, bulk data movement is the most common case, and there are tried and true ways of doing that without the latest SOA secret sauce.

The moving parts of the simple-enough solution are:

Continue reading "OWB Integration with Salesforce.com Data: Sometimes The Old Ways are the Best Ways" »

January 24, 2009

OWB 11.2 in Database Beta: Looking for a Few Good Testers

Oracle Warehouse Builder 11.2 is currently in beta testing, as part of the Oracle Database 11.2 beta program. We are especially interested in feedback in the following areas:


  • importing an OWB 10.2 or OWB 11.1 repository into OWB 11.2

  • running mappings originally designed and then imported from OWB 10.2 or 11.1

  • testing out our new orphan management functionality for dimensions

  • using our new UI for promoting OWB repository from dev to test to production (aka Multiple Configurations)

  • Testing our new UI client on Linux


The current beta is chock-full of other new tricks that you may have heard about but you'll have to get into the beta program to try out. If your company can sign an NDA, are existing users of OWB, and can test the above mentioned features, please contact OWB PM at dwhdoc_us@oracle.com for more information. Put BETA INVITE in the subject line, followed by your company name.

Be sure to tell us as much as you can about your company, your current OWB and database versions, your current and anticipated OWB use cases-- data warehousing, general data integration, data quality, or MDM-- your data volumes, and your sources and targets (Oracle, non-Oracle, ERP applications, or what have you).

January 26, 2009

Forwarding: Exadata DW Webcast with Senior VP Juan Loazia

Hi, all,

OWB is of course used by customers other than DW customers, but I thought this announcement would find interested DW users here. An upcoming webcast on Oracle Exadata will provide a good close look at how Oracle provides disruptive improvements in data warehouse query performance through eliminating bottlenecks inherent in traditional architectures.

This talk is open to anyone, and provides a great way to get past any FUD you're hearing from DW appliance vendors who may be experiencing a bit of discomfort at the moment.

You can read through the abstract to find out what's on the agenda-- or you can just do as you're told and sign up now at the following link:
https://www.techwebonlineevents.com/ars/eventregistration.do?mode=eventreg&F=1001323&K=1AA1A1

CMP Webcast: Extreme Performance for Your Data Warehouse
Wednesday, January 28th, 9:00 am PST


Data warehouses are tripling in size every two years and supporting ever-larger databases with ever increasing demands from business users to get “answers” faster requires a new way to approach this challenge. Oracle Exadata overcomes the limitations of conventional storage by utilizing a massively parallel architecture to dramatically increase data bandwidth between database and storage servers. In this webcast, we’ll examine these limitations and demonstrate how Oracle Exadata delivers extremely fast and completely scalable, enterprise-ready systems. In addition, you’ll learn how Oracle Exadata can improve query performance for data warehouses by at least a factor of 10x.

Featured Speaker:
Juan R. Loaiza, Senior Vice President, Systems Technologies -- Oracle Corporation

Juan has been at Oracle for over 20 years and manages "systems" aspects of database development including: High Availability, RMAN, Data Guard, Oracle Secure Backup, Data and Index Management, Flashback, Performance, Streams, AQ, Exadata, Virtual Operating System, Platform Specific Development, CORE, and SAP Development.

Registration Information

Register at:
https://www.techwebonlineevents.com/ars/eventregistration.do?mode=eventreg&F=1001323&K=1AA1A1

January 28, 2009

Forwarding: Webcast Announcement -- Going Beyond ETL, Next Generation Data Integration and Management

We in OWB know that our customers have use cases other than those for which they use OWB today. So some in our audience may want to attend this upcoming webcast about Oracle’s Data Integration Suite offering. Please feel free to forward this to others in your organization who may be interested.

Register for the webcast here:
http://www.oracle.com/dm/09q3field/24440_ev_int_sem_etl_nxtgen_feb09.html


Going Beyond ETL, Next Generation Data Integration and Management

What if you could go beyond conventional ETL to…

  • Consolidate data for improved scalability and performance?
  • Ensure that your data is clean, consistent, and trustworthy?
  • Make better decisions using up-to-date information?
  • Better re-use data as information services?
  • Leverage the IT infrastructure to make better use of what is already in place?

Join this Internet Seminar to learn how companies around the world have gone beyond convention to embrace Oracle Data Integration solutions for unmatched performance, ease of use, flexibility and lower total cost of ownership.

Learn how to promote your data to re-usable information, service-orient your data-centric architecture using Oracle’s Data Integration Suite, and how to solve critical concerns regarding data integration issues that Enterprise Architects, Data Architects, and IT Management face every day.

Monday, February 9, 2009
9:00 a.m. PT/ Noon ET

At the event, you will:

  • Hear about SOA and Data Integration technologies bridges critical information gaps
  • Listen to customer success stories that can help you achieve the results you need
  • Learn about Oracle's strategy for Data Integration and specifically how you can better realize the benefits of going beyond ETL.

Register now for this event.

Data Quality is Job Number 1

Here's a good read from Enterprise Systems Journal-- a reminder of why data quality is important to all of our efforts, and why OWB has extensive data profiling and data quality features built in.

Of course, if you're reading this blog, you're already mindful of these issues, and are using OWB data profiling and data cleansing with all your data and auditing quality regularly... right? :)


Analysis: Data Quality is Job Number 1

BI tools are only as good as the quality of the data they work with. Analyst Michael Schiff is still surprised at how many BI professionals still ignore this fact.
By Mike Schiff
1/28/2009
Much attention has been focused recently on integrating data from multiple sources to populate data warehouses or data marts for analysis purposes or as part of a migration effort for new enterprise applications. For example, a recent press release from a well-established business intelligence vendor highlights the ability of its BI platform to access multiple data sources where they reside without first having to move the data into a data warehouse or a data mart.

Rather than dwelling on the tradeoffs (perhaps a topic for future analysis) among centralized data warehouses, federated databases, enterprise information integration (EII), or even the old concept of virtual data warehouses, I would like to point out a common property they share: the quality of the information obtained from any of them is directly dependent on the quality of the data they access or contain. In other words, GIGO (garbage in, garbage out) always has applied (and always will apply) to both analytical and operational systems.

We all recognize this, yet I continue to be amazed as to how often we ignore the fact. In most cases, it's not deliberate; rather it results from taking at face value preliminary assumptions about data quality rather than using techniques such as data profiling to validate the assumptions.


About January 2009

This page contains all entries posted to Oracle Warehouse Builder (OWB) Weblog in January 2009. They are listed from oldest to newest.

December 2008 is the previous archive.

February 2009 is the next archive.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type and Oracle