Web Sites

• Oracle SOA Architect Center
• Oracle SOA Suite Home Page
• SOA Specs Information (Thomas Erl)

Blogs

The following bloggers I find useful

• My Blog :-)
• Matt Wright
• Ramkumar Menon
• IT Eye  (often Mike van Alst on SOA)
• Mike van Alst
• Clemens Utschig
• Marc Kelderman
• SOA/BPM PM Blog (often Clemens Utschig)
• Richard Veryard

Books

I have created a book list on Amazon.

Navigating to your Process

Obtaining the WSDLs of a process is very easy.  Log in to BPEL console and select the processes tab.

Then select the desired process.

This will display the process details.

Obtaining the WSDL Implemented by Our Process

To obtain the WSDL interface to the process click on the WSDL tab.  Ensure that you have the correct version selected.

This will bring up the WSDL associated with the partner link implemented by this process.

Obtaining the WSDLs Called by Our Process

To obtain the WSDLs called by the process then go to the descriptor tab.

This provides a list of all the partner links in the BPEL process, including the partner links implemented by the BPEL process.

From here we can select the WSDL that is needed.   We can also see the properties associated with the partner link.

Note that often the BPEL designer will have created a wrapper WSDL that adds partner link information that is required by BPEL.  If this is the case, and it will be the case for most external WSDL documents, then it is necessary to examine the wrapper WSDL and extract the location attribute of the WSDL import statement.

This can then be plugged into URL of the wrapper WSDL to provide the actual service WSDL.  Usuall this can also be obtained by removing the “Ref” suffice from the partner link WSDL.

This will provide the actual WSDL that is being used by our BPEL process, rather than the wrapper that references it.

Why Bother?

Why go to all this trouble, what is the value in obtaining the WSDLs used by a BPEL process?

Well there are several reasons.  A few are outlined below

• There may be difference in behavior between different environments and we wish to confirm that they are using the same WSDL definitions.
• We may need to create a test environment that requires us to emulate WSDL services provided.
• We may want to check endpoint details to ensure that we are able to navigate through firewalls in our environment.
• We may not have immediate access to the BPEL project and want to check some WSDL interface settings.
• We may just be nosy!

Download the Whole BPEL Process

It is also possible to download the whole BPEL suitcase (packaged process) by selecting the Manage tab of the process we want to download under the Processes tab.

At the bottom of this screen we can download the suitcase by clicking the Export Process button.  This downloads the suitcase as a zip file.

Summary

Often we are looking at deployed processes without easy access to the original JDeveloper project.  Through the BPEL console we can verify both individual WSDL interfaces and also download the whole BPEL source, making it easy to check what is happening.  So remember, next time you are faced with odd behavior and want to look at the BPEL source or partner link details, you don’t need to call the developer!

Target Architecture

I am going to build my 11g cluster on 3 machines.

• Machine DB will host an 11gR1 database.  I will also use it to host a software load balancer (I will use WebCache).
• Machine SOA1 will host two WebLogic installations.  A WebLogic 11g installation will have a single SOA domain hosting a SOA Suite cluster, including BAM.  A WebLogic 10.3 installation will have a single OSB domain hosting an OSB cluster.
• Machine SOA2 will have the same software as SOA1 and will host the same two domains.

When OSB 11g is released then the need for two separate WebLogic installations will go away as the intention is for OSB and the rest of SOA Suite to run on the same WebLogic software version.

As there are two WebLogic domains then I will run an admin server on each machine, one domains admin server on SOA1 and the other domains admin server on SOA2.  This helps reduce the memory footprint.

Logically the architecture is shown below with a load balancer distributing load across the SOA and OSB clusters with a backend 11g database.

As for testing I don’t have a hardware load balancer then I run the load balancer on the same machine as the database to give the physical architecture shown below.

I will run this on 3 virtual machines on a server with 8gb memory, allowing 2gb for each virtual machine.

As a large number of customers seem to be running Linux these days I will use Oracle Enterprise Linux 5.3.  I will use 64-bit Linux for the DB machine and 32-bit Linux for the SOA machines.

It is quite common for clusters to be using a RAC database rather than a single instance database, but that was one VM too many for me to get my head around.

Software Required

So now we have identified the logical and physical architecture we need to identify what software we will require.  The software used is all available for download from OTN by clicking on the software link as shown in the table below.  Our target machine for the software is also shown in the table.

Load Balancing

There are a number of software load balancers available, including functionality built into Linux so why did I use WebCache.  Well there are a number of reasons.

1. I like WebCache
2. It has a nice web based UI for configuring and monitoring
3. It supports cookie based affinity (see previous post for importance of this)
4. It does the job

Just be careful when using WebCache with SOA Suite that you do not use it to cache data.  To my knowledge no testing has been done within Oracle with using WebCache in conjunction with SOA Suite 11g so don’t deploy it in a production environment.

I have to confess that the idea of using WebCache as a load balancer was not mine, but my colleague Nick Cosmidis, so thanks Nick.

Other Resources

Setting up a cluster requires shared storage, ideally for the domain home but also for shared resources such as JMS message stores.  I could have used an iSCSI appliance to provide this but I chose instead to use the DB machine as a shared file server for the mid-tier components.

The cluster also requires IP addresses.  Obvious but there are different requirements for those IP addresses.  The IP address for the load balancer must be routable from all the clients of the cluster.  The database, SOA Suite and OSB instances can have non-routable IP addresses as long as they can talk to each other and the the load balancer.  The clients don’t have to be able to access the database, SOA Suite or OSB directly because they will go through the load balancer.

Virtualization

I am running this on a virtualized environment, a single 8GB machine hosting all three machines.  The only software virtualization fully supported by Oracle is Oracle Virtual Machine.  That is not to say it won’t work on other software virtualization environments such as VMware, just that it is not fully supported on those environments.  For more information on Oracle’s support policy with respect to virtualization in general check out this link.  For specific information on VMware support then check Note 249212.1 in MetaLink.

• Lots of customers are running SOA Suite clusters
• Lots of them haven't read the High Availability Guide (10g or 11g)
• Lots of them haven't read the Enterprise Deployment Guide or EDG (10g or 11g)
• Many of them have problems because of the points above

Part of the problem for many customers is that setting up a cluster has a lot of steps and a few gotchas that can come back to bite you.  Just got off the phone with a customer who was having problems with a cluster install, nothing too serious but irritating and slowing him down.  Unless the HA & EDG are followed very carefully it is easy to make mistakes.  A few common problem areas I have seen are

• Failing to separate the design and run time of the ESB
  The ESB design time is a singleton and there must never be more than one instance of the ESB design time active against the same repository at the same time.
• Poor configuration of JGroups or Coherence
  In 10g JGroups is used to identify cluster membership, in 11g Coherence plays the same role.  If these are not configured consistently across the cluster then one part of the cluster may be unaware of the existence of other parts of the cluster with dire circumstances for some shared resources.
• Failure to set up virtual addresses correctly
  The cluster should have a single virtual address. This virtual address needs to be configured in the HTTP listener, into the OC4J servlet engine and into the BPEL URL settings, amongst other places.  Failure to do this can lead to odd behavior.
• Failure to test on a cluster
  Seems many companies have clusters in production but not in test and dev.  Surely no-one is that stupid you say.  Well they may configure test instances with a cluster but for resource reasons run only one node of the cluster, after all a one node cluster is still a cluster right...  They then wonder why they only get certain problems in production and can't reproduce them in test.
• Many customers fail to have suitable test load balancers
  Often customer will not have a hardware load balancer for use in their test or dev environments and will rely on a software load balancer.  Some of these software load balancers use IP stickiness to keep affinity between clients and servers.  This is bad when testing because it means that if you run a test script from a single machine it will only target a single machine in the cluster....  Make sure when testing with a software load balancer that it uses HTTP cookie affinity rather than IP address affinity.
• Poor testing with BPEL drivers
  Often we use BPEL processes as test harnesses to exercise functionality.  This is good but it doesn't work well in a cluster without some modifications to the driver process.  By default BPEL will optimize communications, within the same JVM it will use Java calls, between JVMs in the same cluster it will use ORMI, it only uses HTTP if it has to.  To test properly we need to distribute load through the load balancer and hence want to use HTTP.  In the invokes we can use the property optSoapShortcut=false to force calls to go through HTTP and hence the load balancer.

The above isn't an exhaustive list.  If you have others then feel free to share them, I would love to add more to the list.

A few weeks ago I was in Orlando on an HA training course for 11g SOA Suite and I'll be sharing some stuff about this in future blogs.  I also put together a 1 day course for support engineers around SOA Architectures and again I will post some of this stuff over the next couple of weeks.

But this week I am in San Francisco for Open World.  If those who know want to stop by and see me I will be in the Support Stars Bar on Monday afternoon and Tuesday morning.  On Wednesday at 12:30 I have a book signing at the bookstore.  So stop by and say hello.

This is my first US Open World and I am looking forward to the chance to catch up with old friends and hopefully make some new ones.

The objective of the three peaks challenge is to walk up the highest peak in each of England, Scotland and Wales in a single 24 hour period.

Peak 1 - Ben Nevis

Ben Nevis in Scotland is the highest peak of the three at 1344m.  We started our ascent at 5pm on Friday and reached the summit by 7:30pm.  As can be seen on the picture of myself and Neil Spink reading the SOA Suite book, weather at the top was clear so that we could see for miles.  The weather was hot and sunny and I struggled a little on the way down but we were all off the mountain by 9:45pm and on the road to Scafell Pike in the Lake District, not far from my brother-in-laws house.  We left for Scafell at 9:51pm.

Peak 2 - Scafell Pike

Scafell Pike in England is the smallest of the three peaks at 978m.  We arrived at 3:32am, just as the sky was lightening up in the east.  The moon was also very bright with little cloud cover so we had no problem seeing our route and we reached the summit by 5:30am just as the sun was rising above the surrounding peaks of the Lake District.  There was little cloud about and Daniel Roberts and myself could easily see the book at the top.  We were all back down the mountain by 7:17pm and on the road to Snowdon.  Our drivers might have been controlling 17 seater mini-buses, but to the occupants we felt like we were participating in a particularly tough rally as we wound our way out through the tortuous lanes on our way to North Wales.

Peak 3 - Snowdon

The final peak, Snowdon, is the tallest mountain in England and Wales at 1,085m and was challenging in two ways.  When we arrived at 11:50am we had already spent more than 8 hours on mountains and more than 11 hours crammed into what seemed to be a shrinking mini-bus, so we were not at our most energetic.  Also at Snowdon the weather was failing and it was beginning to rain.  However arrival at Snowdon energised us and within a minute of arrival we were striking out towards the mountain.  The initial approach up the Pyg Track was fairly easy going, with just a scramble at the end to reach the summit by 1:45pm.  The summit was cloudy and wet so no-one wanted to hang around for long, especially myself and Andy Gale who found the book rather damp going.  The descent was also very quick, using the Miners Track to drop down off the mountain very rapidly, followed by what felt like a long run/walk around the mountain back to the start point.  We had all arrived back by 4:02pm, meaning that as a group we completed the challenge in 23 hours and 2 minutes.

Thanks

Special thanks must go to our drivers, Colin, James, Martin and Jackie who got us between peaks rapidly and safely.  I was very appreciative of Andy Gale who made sure I kept pace on Scafell Pike and Snowdon.  Finally the determination award must go to Neil Spink who despite blisters and damaged knees climbed Snowdon and completed the challenge.

Sponsorship

Apart from wanting to show that we were not all over the hill (most of us were well over 40!) we also wanted to raise money for the NSPCC.  We are still a few hundred pounds shy of raising £10,000 for this charity so please feel free to go to my justgiving page and sponsor me.

Statistics

The following is from a GPS tracker we took with us.

Ben Nevis - start at 51m, Top 1,347m  (include the trig point I guess), distance 15.1km or 9.38 miles

Scafell Pike - start at 29m, Top  985m, distance 9 km or.5.59 miles

Snowdon - Lowest Point 376m, Top 1,088m , distance 12.4 KM or 7.7miles

Total Climb = 1296 + 956 + 712 = 2,964m or 9,724 ft

Total Distance = 22.67miles

• Oracle WebLogic Server 11g R1 (with or without Oracle Enterprise Pack for Eclipse - OEPE).
• Oracle SOA Suite 11g R1 (includes BPEL, Human Workflow, Business Rules, BAM, Mediator, B2B, OWSM, User Messaging Service and Enterprise Manager Fusion Middleware Control)
• Oracle Repository Creation Utility 11g R1 (creates the database schemas needed by SOA Suite and other components)
• Oracle JDeveloper 11g R1 this also needs the SOA plug in which should be available through the JDev extensions mechanism.
• Documentation is also available online or may be downloaded.

Watch the launch webcast and enjoy the latest release of the software, it a big step forward for SOA Suite.

Having spent a couple of weeks working on a SOA Suite cluster thought I would share some thoughts around clustering and SOA Suite.  Clustering of both BPEL Process Manager and Oracle Service Bus is relatively straightforward but there are a few gotchas.  Both BPEL and SOA Suite are stateless in the way they implement clustering, however BPEL does of course persist state to a database.

SOA Suite Clusters

Both BPEL and OSB clusters expect to be fronted by a load balancer.  Both can provide load balancing through a front end web server but a hardware load balancer is the best approach as shown in the diagram.

In this example we have an Oracle Real Application Cluster database running on 3-nodes to provide a high availability database environment.  We then have two clusters.  A cluster of 5 BPEL Process Manager instances, all pointing to the same RAC database, and a cluster of 5 OSB instances.  The BPEL cluster and the OSB cluster are both fronted by two hardware load balancers.

The BPEL instances and the OSB instances are in an active-active mode, meaning that all nodes are processing requests at the same time.  The load balancers are in active-passive mode, meaning that one load balancer processes all the traffic with the other load balancer acting as a hot standby in case of failure of the first load balancer.

This configuration avoids a single point of failure as every component is duplicated.  The system has been sized to be able to sustain the expected load even in the event of losing a machine.  This is known as "n+1" architecture, meaning that we need "n" machines to meet the requirements and so we provide "n+1" machines to allow for machine failure.  In the example shown we actually have two machines more than we need in normal operation because the the RAC database is running an "n+1" configuration as well as the SOA Suite.  Note that by running OSB on the same machines as BPEL we reduce the amount of extra hardware needed for failover and also reduce the latency of OSB-BPEL communication.

In the event of a failure then only in-process requests would be impacted.  Any requests that are "idempotent" (meaning they can be resubmitted with no ill effects) can be set up to automatically retry, further reducing the impact of a software or hardware failure.  Both BPEL and OSB can be set to automatically retry requests in event of failure, making the failover transparent.

Note that clustering does not help if we have a site failure due to fire, flooding, power failure or air conditioning failure for example.  In those cases we would need to have some sort of disaster recovery site, perhaps using Oracle Data Guard to keep the sites in synch at the database level.

BPEL Clusters

A BPEL cluster is effectively defined by a shared dehydration store.  Synchronous interactions must be processed within a single BPEL server instance, as the client has connected to a socket and expects a response on that same socket.  Asynchronous interactions are like any other long running BPEL process and may start processing on one node and then have processing resumed on another node, either due to failure or some other event.  The dehydration store (an oracle RAC database in the example above) provides a common location for process state that allows any BPEL instance to resume execution of a process instance.

When installing a BPEL cluster the best place to start is the High Availability Guide.  This outlines that you create a BPEL cluster by doing the following:

• Get the address of the load balancer.
• Run the repository creation assistant to create the BPEL meta-data in the database.
• Install OHS and OC4J components on the machines.
• Install BPEL Process Manager into the app server instances installed previously.
• If using a RAC database make sure that the JNDI data sources are using all RAC nodes.  See the Enterprise Deployment Guide for instructions on using Fast Connection Failover.
• Configure BPEL in a cluster as outlined in the Enterprise Deployment Guide.
• Set up JGroups to make all nodes aware of each other in the cluster.  Note that in 11g Coherence will be used for this which will simplify configuration.
• Set enableCluster and ClusterName in the collaxa-config.xml file.
• Make sure the BPEL PM instances all use the load balancer address for server URL and callback URLs.  This ensures that in event of node failure requests and responses are rerouted to remaining instances.
• Deploy processes on each node to make sure that all components are available on all nodes.  If you don't do this some processes will work because they don't have dependencies on non-BPEL components, but others will not.

Once set up use the BPEL fault handling framework to make sure that any calls to OSB are automatically retried in event of failure.

OSB Clusters

An OSB domain may have a single OSB cluster.  This cluster is installed like any other cluster in WebLogic.  Details on configuring the cluster can be found in the Creating WebLogic Domains Using the Configuration Wizard documentation.  For normal operation the OSB cluster is completely stateless, however the metrics gathering and aggregation takes place in a singleton service that by default is assigned to the first machine created in the cluster.  If this server fails then metrics will stay in the message queues to which they are delivered until a new instance starts.

• Get address of the load balancer.
• Install OSB software onto each machine or into a single shared location.  If a shared location make sure that the reference to it is the same on each machine.
• Create an OSB domain
• Creating a cluster and machines and assign servers to machines as explained in the Creating WebLogic Domains Using the Configuration Wizard documentation.
• Use load balancer address for cluster address.
• If you are not using a shared file location for your OSB install then you need to copy the contents of the osb domain directory to all nodes.  This ensure that the correct scripts are available for the node manager to launch managed servers.
• Run node manager on each machine.
• You can now launch your admin server and start the managed servers one at a time.  It is recommended that you start the OSB server running the data collectors first.  This will avoid timeouts on the other machines in the cluster and ensure that metrics are available.
• If message queues are required to be highly available then they should use persistent storage either on a shared highly available disk (a SAN for example) or they should use database persistence.
• If you are using an Oracle database then you should configure your OSB domain to store metrics in the Oracle database rather than in pointbase as explained in Creating WebLogic Domains Using the Configuration Wizard.

When calling BPEL or external web services from OSB make sure that you specify retries to allow for node failure.  Intra-OSB calls should be done using "local" transport for efficiency.

Summary

Configuring a cluster is a bit more involved than configuring a single instance, but it is not massively more complicated and it does provide both scalability and high availability.  Both BPEL and OSB scale linearly with increased nodes.  The only limitation on BPEL is the load on the backend dehydration store.  So go ahead, enjoy a cluster!

Just at a customer who has an 11g RAC database that he wants to use for his SOA repository.  If you try and install SOA Suite into an 11g database, it tells you that the database is not supported and the irca configuration assistant fails to find a java library.  11g is a certified platform for SOA Suite 10.1.3.4 so here is how to get it installed.

IRCA

Before installing the SOA Suite executables you need to run the irca script to create the SOA Suite schemas in the 11g database.  The irca script needs to be able to find an ojdbc14.jar file.  This file is not shipped with the 11g database which provides libraries for Java 5 and Java 6 rather than the almost obsolete Java 1.4.  This leaves you with a couple of options;

• Use an Oracle 10g home as your Oracle home, if you have one on the machine, when running irca.
• Copy the JDBC libraries from an Oracle 10g home (<Oracle10g_Home>/jdbc/lib) to the Oracle 11g jdbc location (<Oracle11g_Home>/jdbc/lib).

Having done this then the irca script should run fine and create the ORABPEL, ORESB and ORAWSM schemas for you in an 11g database.  You are now ready to run the SOA Suite installer.

Installer

When you run the installer to create the SOA Suite instance it will fail when checking for the existence of the SOA Suite schemas unless you patch the installer files first.  The way to do this is to download patch 6265268 from MetaLink and follow the instructions which basically require you to modify the install media as follows:

• Replace DBConnectQueries.jar
  • Move
    • <MEDIA>/stage/Queries/DBConnectQueries/8.4/1/DBConnectQueries.jar
  • to
    • <MEDIA>/stage/Queries/DBConnectQueries/8.4/1/DBConnectQueries.jar.pre6265268
  • Copy
    • <PATCH>/DBConnectQueries.jar
  • from the patch to
  • <MEDIA>/stage/Queries/DBConnectQueries/8.4/1/DBConnectQueries.jar
  • Note that the current patch documentation incorrectly refers to an 8.5 directory rather than the 8.4 that actually exists.
• Replace IP_DBQueries.jar
  • Move
    • <MEDIA>/stage/Queries/IP_DBQueries/3.0/1/IP_DBQueries.jar
  • to
    • <MEDIA>/stage/Queries/IP_DBQueries/3.0/1/IP_DBQueries.jar.pre6265268
  • Copy
    • <PATCH>/IP_DBQueries.jar
  • from the patch to
    • <MEDIA>/stage/Queries/IP_DBQueries/3.0/1/IP_DBQueries.jar

You can now launch the installer and there will be no complaints about an 11g database.  Note that if you are installing a SOA Suite cluster then this will need to be done for each SOA Suite instance being installed.

What’s the Point?

As you are making these changes your mind keeps asking why you are doing this.  Apart from being told to do so by your DBAs there are some good reasons for using 11g.  The 11g database is the most manageable Oracle database ever, and several options only work on the 11g database.  11gR1 has been out for a long time now and so from a longevity perspective its best to put new deployments on an 11gR1 platform even if you have a release-1 strategy to avoid “bleeding edge” deployments.

Finally as you can see, it is not that hard to use an 11g database with SOA Suite.

./runInstaller PREREQ_CONFIG_LOCATION=<PATCH_LOC>/prereq –paramFile <PATCH_LOC>/oraparam.ini

PATCH_LOC is the location where you unzipped the 6339508 patch.  With these new parameters the installer correctly recognizes Oracle Enterprise Linux as a supported platform.  Note that OEL needs some specific patches when used with SOA Suite and these need to be installed prior to running the installer.  Check the documentation for details.

I am seeing an increasing number of customers taking up OEL, I think because of the attraction of a single company supporting the whole technology infrastructure stack, OS, AS and DB.  I expect to see more of this still in the future.

This year together with a number of other Oracle UK employees I will be participating in the Three Peaks Challenge in July - climbing to the top of Ben Nevis, Scafell Pike and Snowdon, all within 24 hours.

This will be no mean feat - it involves making your own sandwiches as well as walking for 15 hours/30 miles up/down over 10,000 feet interspersed with 500miles of driving. I've been walking up and down stairs in the office for weeks in preparation.

If you would like to sponsor me in this somewhat madcap activity, then donations can be made via PayPal, Credit or Debit Card at my Justgiving site.  If you’re a UK taxpayer, Justgiving makes sure 25% in Gift Aid, plus a 3% supplement, are added to your donation.

So go ahead, sponsor me!

The native format builder wizard in the file adapter is great at reading flat file structures but doesn’t support reading more structured file structures.  Sometimes we need to read more complex structures such as master-detail records.  Let’s look at how we can use the file adapter to read structured file formats.

For example imagine a laundry list file such as the one below:

P,101,James L,Shirt,2,Starch L,Socks,6,De-odorise L,Pants,2,Remove Stains

P,220,JoJo L,Sweatshirt,1,Handwash L,Shirt,3,No Iron L,Socks,2,Iron L,Pants,2,Steam Press L,Tie,1,Dry clean

P,305,Ruth L,Skirt,7,Iron Pleats L,Socks,8,Iron

To make it easier to see the record structure I have put boxes around the individual records.  This can be represented in XML as shown below

The native format builder wizard would either treat each line as either the same type of record (“Multiple records are of single type”) or as two different types of record (“Multiple records are of different types”).

In the first case we would have a single record type with four fields that fails to distinguish between people (lines pre-fixed with ‘P’) and laundry items (lines pre-fixed with ‘L’).  This does not reflect the fact that the two lines are of different types.  The second case, multiple records are of different types, is closer to what we want.  It creates two records types, the type being determined by the value of the first field.  The XML for this is shown below:

However this case creates a list of two record types with no recognition that one record is nested inside the other.

Comparing the two XML representations we can see what needs to be done; we need to add an Items element under Room and move the Item element to be under Items.  The question is how do we describe this using the native schema constructs supported by the file adapter.

Creating a Master-Detail Records Native Format Schema

The easiest way to deal with this is to use the native format builder wizard in the file adapter to create the basic outline of the records for us.  We choose “Multiple records are of different types” and provide appropriate names for the two records types and individual fields within the records.

Having created the basic native format schema we can now edit it to be exactly the way we want it for the laundry list.

We make the following changes in the generated native schema file:

• Replace the <choice> element by a <sequence> element.  In conjunction with other changes this will give us a list of elements of the same type, rather than of list of elements of two different types.

  • <xsd:choice minOccurs="1" maxOccurs="unbounded"     nxsd:choiceCondition="terminated"     nxsd:terminatedBy=",">     … </xsd:choice>

    <xsd:sequence>    … </xsd:sequence>

• Replace the conditionValue attributes with startsWith attributes and add a comma to the end of the attribute values.  This will allow the native schema processor to identify the start of master records and child records.  We also add a maxOccurs attribute to the elements modified so that they can have multiple instances in a sequence.

  • <xsd:element name="Item"     nxsd:conditionValue="L"> … </xsd:element>	<xsd:element name="Item"     nxsd:startsWith="L,"     maxOccurs="unbounded"> … </xsd:element>
    <xsd:element name="Room"     nxsd:conditionValue="P"> … </xsd:element>	<xsd:element name="Room"     nxsd:startsWith="P,"     maxOccurs="unbounded">     … </xsd:element>

• Add an <Items> element as a sequence to the <Room> element sequence and move the <Item> element to be inside the <Items> sequence.

  <xsd:element name="Room" nxsd:startsWith="P," maxOccurs="unbounded">
      <xsd:complexType>
          <xsd:sequence>
              …
              <xsd:element name="Items" maxOccurs="1">
                  <xsd:complexType>
                      <xsd:sequence>
                          <xsd:element name="Item" nxsd:startsWith="L," maxOccurs="unbounded">
                              …
                          </xsd:element>
                      </xsd:sequence>
                  </xsd:complexType>
              </xsd:element>
          </xsd:sequence>
      </xsd:complexType>
  </xsd:element>

This gives us a native schema format that looks like this:

Note that this is the structure that we were aiming for in the first place.

Sample Code

I have created a simple BPEL process that performs the following steps

• Read a laundry file using the wizard generated native schema format.  This creates an XML document with two distinct records types.
  • Note that in the process I do not delete the file so it is important to make sure that if you re-use the same file with the process you need to update its timestamp!
• Read the laundry file again using the modified native schema format.  This creates an XML document with master-detail style records which reflects the actual format of the file.
  • Note that I use a synch read with the filename provided by the inbound header from the previous file read.   I delete the input file after reading it.
• Write the laundry file using a pure XML schema to create an XML document file of the laundry list.

Sample code is zipped up as a JDeveloper project and can be downloaded here.  In addition to the normal BPEL artifacts within the project, I have included a sample laundry file (laundry.sample.txt) in the top level of the project directory.  After deploying to a BPEL server the process will look in C:\FileTransfer\InBound for the input file, after processing the input file it is written to C:\FileTransfer\Processed and the output file is generated in C:\FileTransfer\OutBound.  Either create appropriate directories or edit the project to use new directories.

Documentation

This entry has used the facilities of the native format schemas in SOA Suite.  These are documented in chapter 7 of the Oracle® Application Server Adapters for Files, FTP, Databases, and Enterprise Messaging User's Guide.

Good luck in using the adapters to process master-detail records!

Last Wednesday the Open Group made available their SOA Source Book.  They describe it as “a collection of source material produced by the SOA Working Group for use by enterprise architects working with Service-Oriented Architecture”.  Having looked at it I have to say it seems to have a high information density and would be a good place to start getting someone's head around SOA concepts.  Well worth a look.

Tuxedo can be considered as the original and purest service oriented architecture.  The key abstraction in Tuxedo is the service and everything is made to fit into the service mould.  It seems strange then that people think of Tuxedo as a legacy application.  Tuxedo is highly regarded by the senior management team in Oracle who view it as a key tool to support extreme transaction processing.  The question is then, how does this relate to the rest of the SOA world which does not subscribe to the Tuxedo technologies such ATMI, C++ or COBOL.

The diagram below shows the different interfaces into and out of the Tuxedo world.  Lets look at them briefly and how they relate to the rest of the SOA world which is focussed on XML, SOAP and HTTP.

Native Client Interfaces

The native client interfaces to Tuxedo are the C, C++, .Net or COBOL client interfaces using ATMI (Application to Transaction Monitor Interface).  There is also a version of ATMI for Java, called Jolt.  These interfaces allow clients to invoke Tuxedo services and get responses.  They do not allow Tuxedo to invoke services in these clients except by listening on Tuxedo message queues, these interfaces are asymmetric.

Legacy Client and Server Interfaces

A Tuxedo domain can interface to other Tuxedo domains, treating their services as though they were its own services.  This capability is extended to other systems such as mainframe systems.  The external systems see Tuxedo services as native services and invoke them as they would any other service, similarly Tuxedo sees these external systems as native Tuxedo services and invokes them as it would any other service.  This provides relatively seamless integration between legacy environments and Tuxedo and allows either side to operate as a server to the other, in other words these interfaces are symmetric.

Open System Interfaces

In addition to treating legacy mainframe interfaces and other Tuxedo domains in the same way as local services Tuxedo can also do this for such open system standards as CORBA and java code running in WebLogic Server.  CORBA applications can invoke Tux services through the CORBA API and Tuxedo services can invoke CORBA objects.  The WebLogic Tuxedo Connector (WTC) extends the capabilities of Jolt to become fully symmetric in that EJBs in WebLogic can be invoked as services from Tuxedo.

A Transactional Note

Note that all the interfaces we have spoken about so far are transactional, as in they are part of the Tuxedo transaction infrastructure, invoking a remote mainframe transaction may cause an XA transaction to be started within Tuxedo.  When calling an EJB in WebLogic this also is part of the overall Tuxedo XA transaction infrastructure.

Web Service Access to Tuxedo

There are two alternative ways to get Web Services to access Tuxedo.  The most obvious is to use SALT (Service Architecture Leveraging Tuxedo) which exposes Tuxedo services as web services, and allows Tuxedo to invoke Web Services as though they were Tuxedo services.  This is a symmetric interface and takes care of all the XML to Tuxedo translations but it is not transactional.  The web service call is not part of the transaction.  A web service request to Tuxedo may cause a Tux transaction to be initiated, but webs services don’t currently provide a transactional context.  Similarly when Tux makes a call to a web service, that call is not part of any Tuxedo transaction.

So what if you want transactionality and access to web services.  This is where the service bus comes in.  The Oracle Service Bus (OSB) takes advantage of the WebLogic Tuxedo Connector to provide a fast efficient and transactional interface to and from the Tuxedo world.  This allows a pipeline to make two seperate calls to Tuxedo as part of the same transaction.  Note that there are a couple fo wrinkles to making this happen and I will deal with those in a later post.

Summary – Long Live Tux, King of Services

Not only is Tuxedo the original service oriented architecture but despite being more than 20 years old, through SALT and WebLogic Tuxedo Connector it still speaks the modern lingo of service buses, Java, XML, SOAP and HTTP.  So if you have a Tuxedo investment don’t write it off, but look at how you can more easily make your Tux services available to the new fangled XML based web service world.