begin
owbsys.wb_workspace_management.set_workspace('', '');
owbsys.start_enable_owb_roles;
end;

Now you can do what you need in direct access tools like SQL Plus...

Windows-32
Linux-32
HP-Itanium
Linux x86-64 (AMD64/EM64T)
HP-UX PA-RISC (64-bit)
Sparc64

All of these can be downloaded from metalink (https://metalink.oracle.com/) under patch id 7005587. You will need to log into metalink to download the patch.

The following platforms will follow shortly:

AIX5L Based Systems (64-bit)
Vista-32 bit (and Windows2008-32, Windows2008-64)
Windows-64 Itanium

More on those platforms once available.

So, lets say we are trying to set up an OWB system, which may be a DW or Data mart. What hardware do we run this on so that we get some half decent performance? Good question. Ask the hardware guy, and you'll get lots of cool stuff that you may not need (or may, but find to expensive). Ask the software guys, and you'll get puzzled looks. Or a pointer back to the hardware guys.

Now what do you do. Well for starters, look at this page: http://www.oracle.com/solutions/business_intelligence/optimized-warehouse-initiative.html

Oracle has addressed the issue with pre-defined configurations that create a BALANCED system. It is not a system that will give you infinitely great performance for the smallest $$$ amount... it is a balanced system. This means you will get a good throughput in a normal data warehouse like situation. It does this balancing because it specifies all components. So not just CPU and memory, but disks, HBAs (connect your CPU to your disky things and is kind of required to actually load data and process it - yet often forgotten).

Back to sizing for OWB, these systems will do a good job because they are balanced, and we would expect them to be a good starting point for any DW system.

Since OWB does the ETL in the database, you would not need any extra hardware to host the ETL server, it will go into this single system. Due to that, sizing is simpler and hardware utilization is better.

For data quality sizing is a bit more complex, but even here, if you look at the basic systems, sizing a profiling application is roughly equal. Look at the total data size, determine complexity (much less users on the profile run, a few doing simple queries after it is profiled) and these configs would work as a good starting point as well.

Looking at that optimized warehouse page, note that most vendors are present, so you can kind of pick the one you like. Now I know the temptation is there to skimp on stuff, but if you start with this, and don't skimp on items, you might just have sized your system about right...

The beauty of using ABAP and SAP JCO components is that it does not matter! To OWB it is completely irrelevant what source database is used underneath SAP R/3. It all works outside of the database and there is no SQL being used whatsoever. That makes the solution so nice and so comfortable for SAP administrators. It is all in SAP, adhering to SAP rules and using SAP code.

So whether SAP runs on SQL Server, DB2 or Oracle (or anything else), you can use OWB to extract data from SAP.

Now, if SAP runs on Oracle, and not many people know this, you can treat the system as Oracle as well. So you can do what many (or most) other ETL tools do and go to the SQL layer. Not a problem. However I would recommend not to do this, and any ETL tool that uses SQL to access SAP is destined to become a problem. SQL looks at database tables, not pool and cluster tables. These are SAP constructs, not database constructs. Some of the most important tables in SAP are non-transparent (e.g. pool or cluster). OWB approaches these tables using ABAP so it can extract from them, no problem!

However in order to view data inside OWB using the public views it is necessary to have this set and as OWB creates the connection, the only solution is a logon trigger. This also applies if you are going into any system where you do not control the login. In SQL Plus, you simply run the procedure after logging in or do some session stuff...

As the trigger requires a pl/sql procedure that has execution rights on owbsys.wb_workspace_management.set_workspace this
needs to be granted to the user. In addition, the OWB roles need to be enabled.

The workaround for this is to create the following procedure, grant the following rights to the users and then the logon trigger….

Note the logon trigger has to be granted by sysdba.

Package Script

CREATE OR REPLACE procedure wb_initialize_workspace
authid current_user as

/* To return the first workspace owned by a user */
CURSOR ws_user_owner IS
SELECT MIN(workspace_name) KEEP (DENSE_RANK FIRST ORDER BY workspace_id), user_name
FROM owbsys.all_iv_workspace_assignments
where user_name = USER and isworkspaceowner = 1
GROUP BY user_name;

/* To return the first workspace associated with a user */
CURSOR ws_user_user IS
SELECT MIN(workspace_id) KEEP (DENSE_RANK FIRST ORDER BY workspace_id)
FROM owbsys.all_iv_workspace_assignments
where user_name = USER
GROUP BY user_name;

p_id number;

/* To find the owner of a given workspace */
CURSOR ws_owner_user IS
SELECT workspace_name, user_name
FROM owbsys.all_iv_workspace_assignments
where workspace_id = p_id and isworkspaceowner = 1;

p_ws varchar2 (32);
p_user varchar2 (32);

BEGIN

/* First try to find a workspace owned by this user. If found make that the active workspace */
open ws_user_owner;
FETCH ws_user_owner INTO
p_ws,
p_user;

IF not(ws_user_owner%NOTFOUND) THEN
owbsys.wb_workspace_management.set_workspace(p_ws,p_user);
close ws_user_owner;
ELSE

/* If none owned, get the first associated with this user */
open ws_user_user;
FETCH ws_user_user INTO
p_id;
IF not(ws_user_user%NOTFOUND) THEN

/* Now find the owner of this workspace */
close ws_user_user;
open ws_owner_user;
FETCH ws_owner_user INTO
p_ws,
p_user;
IF not(ws_owner_user%NOTFOUND) THEN
owbsys.wb_workspace_management.set_workspace(p_ws,p_user);
close ws_owner_user;
END IF;
END IF;
END IF;

END;
/

Grants script

define user = &1
grant execute on owbsys.wb_workspace_management to &user;
grant execute on owbsys.wb_initialize_workspace to &user;
alter user &user default role all except OWB$CLIENT;

Trigger script

set concat ~
define user = &1
CREATE OR REPLACE TRIGGER &user~_LOGON_TRIGGER
AFTER LOGON ON &user.SCHEMA
BEGIN
owbsys.wb_initialize_workspace;
END;
/

In addition when using PL/SQL to query the views, for any design time views you will need to grant select access to the user from owbsys (as roles don't work from within pl/sql). This is of course outside of the trigger stuff discussed here...

Some background, OWB comes with an SAP connector, and has done so for its entire life. We have invested quite a bit of time, effort and knowledge into this capability. Not just from our development staff, but also leveraging SAP expertise from New Frontiers (www.newfrontiers.com). Active users have worked on the connector to make sure it addresses things like the above described need of managing ABAP code in SAP (more later in detail of course). New Frontiers actually created an entire data warehouse out of the box based on the Oracle stack with OWB as the tool to create all ETL from SAP into Oracle and using OWB to model the entire dimensional model. More on that is here.

So what is the problem? In general, you cannot move some custom ABAP into an SAP production system without the SAP administrator actually moving it. So since OWB deploys into the system, that would not be acceptable as the program gets "injected" from the outside. OWB generated ABAP therefore must play within the rules.

Here is how this works in a real environment:

In the development environment you create your SAP location, and deploy the mappings. The system is open, so you run this directly from OWB. Not a problem. Now to move anything into the QA and Production systems, you cannot use the OWB deploy of a mapping (which will go into the SAP app and throw in a custom document - no SAP admin likes this).

QA / Production:

Once you have deployed the actual program into the DEV environment, you will have the SAP admin transfer (that is an SAP term with an SAP application screen) the program to the QA or Production system. While he/she transfers the program, variables can be changes like the data file location etc. These are specifically set up as variables (I forget the actual SAP name for a variable) in our generated ABAP for this purpose. Now the program is correctly loaded and moved without other modifications into the SAP application.

Some customers now want this executed from within SAP, some do still want this executed from within OWB. In the latter case you create a location pointing to the actual SAP production system, and in that location you tell us to use a function module Z_blabla. This function module is custom and we have example code for this. You also tell us in configuration that you do not want to deploy, but just want to execute the ABAP. Now OWB skips deployment (the actual Report is already there) and just executes the transported ABAP from its new location. The data file is generated, OWB will FTP the file and load it.

The simple way of getting OWB to use a dblink is to create your own connector (you may have to delete the one OWB creates by default when creating a mapping!) and specify a dblink on it.

My dblink is created as orcl@tgt_to_hr. It is important to name the connector the same as the dblink without using the servicename entry. So my connector was named TGT_TO_HR.

Once the link is in the db, and you are creating the connector, place the link name in the Details section. Make sure to use Browse... at least you are sure you have the right link name.

Once you have done this, the generated code will show the dblink being used and uses the schema name (as we saw earlier). Deploying the code will now use the precreated dblink.

There are roughly three ways of doing this in OWB:

1) Configure a mapping and set a dblink and schema name property on each mapping (note is not the recommended way)
2) Supply a database link name on the connector between the target module and the source module
3) Use a DBLink type location for the source

Since the third one is most complex one, but used in the EBS cases in house, I figure, let's start with that one.

First thing to do is create the DB link location itself (it will specify the database link to use and the schema etc.)

To explain this dialog a little bit more, the From Location is the target location in which the mapping will live, the database link is the name of the db link as found in the target schema. The dblink of course connects to your source schema as a normal db link.

Next you would associate the dblink location you just created with the SOURCE (!) module, as shown here, and then set the configuration for the source module to use this dblink location.

Note that you cannot use the DBLink location to import metadata.

This completes the work on the source and on the dblink location.

Once you have done this, you have associated the target with the source because in the dblink location you specify the from (target) and now by adding this to the source, the to is also known. OWB now generates you a connector on the target using the dblink. Open the connector to see the actual dblink details as specified on the dblink location.

Since this target location is already registered, all info above is grayed out...

Next is the target, where nothing else is really required. You create a mapping from source table to target table and generate code.

As you can see in line 337 the source schema we specified in dblink location and the dblink are taken into the generated code....

Now that you are using this setup, your credentials to for example production schemas are hidden behind the dblink. A DBA will set up the link, you use it without knowing the details for production data.

I'll see if I can quickly add the #2 option at a later point. It is a bit simpler, in that you on the connector override the database link name with your own. However it allows the source to still use full credentials and therefore potentially gives data access to the wrong people.

BTW, not what I was thinking when I started but you can apply the same to a "remote target". Simply deploy the mapping in what I have as a target here. Create a remote location and a dblink loc for target (e.g. data going to that). Drag that operator into the mapping and now you go DBLink two ways...

Great for writing to other DBs like SQL Server of course...

Here we have a simple mapping, where I join on code. I want to get all records (yeah not a great business case I know) from both tables, so I know if there are records on either side without matches.

My join looks like this:

Now if I complete the mapping to look like this, I can generate code.

As you see by default (this is OWB 10.2.0.3) you get an ANSI join syntax:

Want something not ANSI, go to configuration on the mapping and uncheck the ANSI SQL Syntax code option. Note that if you are doing Multi Config, you can change this per deployment of the mapping...

We did ODTUG, which was my 5th conference in a row, and as always, great fun and a very good conference. I especially liked the new Hyperion track at ODTUG. With the Oracle family growing I think there is a lot of value provided by the conferences in introducing everyone to the new technologies in Oracle's portfolio. I had the privilege to be on a panel and discuss technology together with Mark Rittman (you can read more on his blog) and some other folks. Good fun, quite interesting to discuss end-user vs. IT perspectives.

One of the things I took away from that is that we need to bridge that gap in the near future. We need to enable end users to get their information when they need it, but I think we need to not forget architecture and infrastructure. Yes I know I'm one of the old timers who think that is important.

End users should have access to data, but in a controlled manner. At least that is what I think. The Hyperion tools allow the finance guys to set up their own systems, manage their own hierarchies, which is great. But it does not mean that we can't build this on top of a scalable and secure infrastructure.

So for Warehouse Builder I think we will look more and more into enabling more business users to get to their data. Let's call it end user data integration. That way we can allow for data quality to be embedded in everything that is done, we can centralize metadata without being obnoxious about it, and we can secure the base data.

Anyways these are some of my thoughts around direction for OWB in the long run. In the mean time, ODTUG is done and we are already looking at OpenWorld in September. Bigger, better and more involved as always.

Openworld will showcase at least 5 sessions on OWB, a handson session and of course demo stations for data integration and data quality. We will have product management and development around, so if you are a current user, this is the event to discuss your problems, solutions and ideas with the guys and girls who build the product. And of course, San Francisco in September is a nice place to be... So see you in San Francisco!

Expect new postings to start appearing over the next days again. So stay tuned.

Warehouse Builder depends on infrastructure in the Oracle Database for most of what it does... including calling external Web Services. Briefly, this requires that you install JPublisher in your database, which at this point means that you need to install SQLJ from the Oracle Database Client CD, as well as the Web Services Callout Client.

A lucid and concise description of how you configure an Oracle 10gR2 Database to consume Web services was posted in this Oracle By Example (OBE):

http://www.oracle.com/technology/obe/10gr2_db_single/develop/java/java_otn.htm

It takes you through installing all the required components and then calling an example web service from PL/SQL.

Once you have this working you are ready to start using calls to external web services from OWB (of which I shall write more in a future post).

Sometimes OWB PM gets questions about ways to use Warehouse Builder in consulting situations, sales-related proof of concepts implementations, and so on. Some that came up recently include:

• How OWB can be used to create or consume Web Services (a subject of a recent whitepaper);
  
• How OWB can send email with attachments (the process flow task for sending email makes no provisions for attachments);
  
• How OWB can make updates to an Active Directory or other LDAP server

In the second case, we recommended the PL/SQL package UTL_MAIL.  Incorporating a short bit of PL/SQL code that called the needed functions (make the attachment, create the mail message with the attachment, send it out) into a process flow would cover the customer's/developer's need.

The third case was more interesting, because of the thinking behind it, and the questions it raised.

The original question that reached PM was, "Can OWB write to ODBC targets? (Or do I need a different ETL product?)" The line of thinking was something like:

• I'm using an ETL tool
  
• An ETL tool can update data in database targets
  
• Therefore, if I want to update something that isn't a database, I should make it look like a database target
  
• Therefore, the customer needs to license a third-party ODBC driver for LDAP, and choose an ETL tool that can write to ODBC
  
• Can OWB write to ODBC? ("Yes, we can", by the way-- the core product can use the gateway for ODBC with any target, no cost.)

This chain of reasoning is quite understandable, but it misses an important point: OWB is much more than a tool for talking to data sources and targets. OWB is now and has always been a great tool to maximize productivity for developers of PL/SQL-based data integration solutions on top of the Oracle database. (Future mechanisms for heterogeneous data integration in our roadmap don't obsolete any of the strengths of OWB when used with Oracle databases.)

The PL/SQL part is crucial here. From OWB you have access to all the functionality Oracle builds into the PL/SQL packages that come with the Oracle database. (This, for example, is how we have incorporated Web Services/SOA functionality into OWB since our 9.2 release. PL/SQL is generated for OWB mappings, and all PL/SQL packages can be accessed as web services, therefore OWB mappings can be exposed as web services. Likewise, PL/SQL can consume Web services, OWB generated code is PL/SQL code, therefore OWB can consume web services.)

In this instance, OWB access to PL/SQL enables a much more direct solution, with no third-party component and no need to abandon OWB. We proposed a little PL/SQL code to call the DBMS_LDAP and DBMS_LDAP_UTL packages from PL/SQL to update the LDAP server directly, and incorporate that code into a process flow.

Can we do more over time to package up capabilities like the ones above? Yes, we can-- whenever the market demands it. (Tell us what you want-- you, after all, are the market.)

But the next time you look in your palette and you can't find the right icon to drag out to do exactly what you want, don't give up, reframe the question: "Can the Oracle Database platform do this? Are there PL/SQL packages that can do this?" You won't want to or need to do this often-- but when you choose to, yes, you can.