Sunday Mar 21, 2010

moving blog

Its seems to be quite popular for Sun people to be shifting blog address and I'm doing the same.

Now that OpenESB has been made "non-strategic" and while we wait for the legal processes that take place during the acquisition of global companies, I've got some more time to spend working on software architecture in general. So I've started up an another blog where I can keep track of my ideas.

I have no idea if anyone regularly follows this blog, but if you are interested in another viewpoint on software architecture ideas then feel free to come on by.

If I end up with a job at Oracle, and one that provides something worth writing about, then I'll resuscitate this site.

Tuesday Nov 10, 2009

Clustering GlassFish ESB - BPEL Failover and Load Balancing

Tom Barrett has created another of his detailed OpenESB tutorials - this time for BPEL HA. It shows how failover of process instances works in a highly available BPEL Engine configuration using a Glassfish Cluster.

For those who would like to follow a screencast, I've recorded the tutorial. The written tutorial was developed with GlassfishESB v2.1 on WinXP. The screencast uses the same version but on OpenSolaris 200906 running in VirtualBox.

Part 1: Establishing a GlassFish Cluster

  • Setting up Glassfish in a cluster and configuring the BPEL engine for persistence and failover

  • Screencast (16mins, swf)

Part 2: Creating the BPEL Test Application

  • Creating a simple bpel process to test failover and deploying it to the cluster

  • Screencast (10mins, swf)

Part 3-4: Establishing the Test Cases and Testing BPEL Failover in the Cluster

  • Using SoapUI to create test suites for the individual process engine instances

  • Running the test suites and monitoring the state of the process instances during process failover and recovery

  • Screencast (10mins, swf)

Part 5: Experiencing Load Balancing to the BPEL SE

  • Configuring the Glassfish Load Balancer plugin with the Sun Web Server to distribute load between the instance of the cluster.

  • This project was not recorded because it is mainly concerns installation and settings in configuration files and is well-covered in the written tutorial.

Sunday Sep 20, 2009

Deploying BPEL Apps to Fuji M8

Deploying apps from GlassfishESB v2.1 to GlassfishESB v3.

There's lots of new functionality being implemented for GlassfishESB v3 as part of the Fuji research project. However a very important consideration is that existing projects that run in the current release (v2.1) will continue to work in the next major release (v3).

This how-to shows how to take a simple BPEL project from GFESBv2.1 and deploy and test it in the latest milestone release of Fuji (M8).

Play Screencast

I had to rush through the screencast very quickly to fit in the 5 minute limit imposed by Jing, so here's a run-down of what is happening

  • Start in the GlassfishESBv2.1 environment
  • I already have a simple service assembly with a HTTP BC and a BPEL SE Service Unit
    • The BPEL SU simply returns "Hello <string>" for whatever string is passed to it
  • Run the unit test to show it working
  • The Fuji runtime does not yet support Application Variables, so I replace the variable with the hardcoded value for the HTTP Port
    • Rebuild the application with the hardcoded value so it can be deployed to the Fuji runtime
  • Stop the GFESBv2.1 server so the Fuji Server can be started and use the same ports.
  • Milestone8 of Fuji requires the latest version of NetBeans, so start up NetBeans 6.7.1
  • The current fuji installer doesn't include BPEL SE, so you'll need to download and install it separately (plus the saxon library it depends on)
  • > curl -O
    > curl -O

  • Start the BPEL SE and HTTP BC components
  • Now install the zip file for our BPEL application Service Assembly.
    • start the application
  • Check the the same URL for the WSDL file to ensure the server is running and the service is available
  • To test the project, I've installed the SOAPUI netbeans plugin.
  • Create a SOAPUI project, point it to the service, and test.
Screencast (again).

Monday Apr 20, 2009

IASA Norway event

After a few years in hibernation, the Norwegian chapter of the International Association of Software Architects is getting back together.

The topic for the first meeting is, "What is the Architect´s Responsibility?". Its partly inspired by the recent "97 Things Every Software Architect Should Know" book.

I´ll be doing a lightning talk on "The responsibility of getting the Concept right", which will be based on Fred Brooks´oft-quoted blurb from his No Silver Bullet paper:

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor or representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.

It should be good to get back into general Software Architecture ideas again after so much specific OpenESB work. If you´re in Oslo come along The Scotsman at 18:00 on Wednesday night.

Wednesday Mar 25, 2009

Constraints on BPEL correlation: A workaround

In a previous blog entry I discussed limitations of BPEL Correlation when performing asynchronous communication within the JBI container.

Paul Perez has provided a workaround for those situations using a BPEL Bridge

This workaround works because the BPEL engine only enforces the Recoverability Systemic Quality for processes that are labelled as Atomic. If the BPEL Bridge is not atomic then it returns an ACK immediately and the original process can continue executing while waiting for the Callback.

The same thing would work if the JavaEE SE was able to differentiate between transactional and non-transactional EJBs. That is, only enforce the Recoverability Systemic Quality on EJB Service Units that are annotated as transactional. That way non-transactional EJBs would result in the ACK being returned upon receipt of the msg and not upon successful completion. (An RFE has been filed for this change).

Thursday Mar 19, 2009

bpel correlation with an EJB WS

Following the previous article on Contraints when using BPEL Correlation with OpenESB, I thought I should provide a little example of how communication with asynch services and correlation can be done from a bpel process. 

This is a simple example I whipped up for a customer workshop.

It starts with some attached predefined XSDs and WSDLs to represent the interfaces:

  • in-out (req-resp) process interface
  • in-only asynch service interface
  • in-only callback interface

  • The process implements the process and callback interfaces.
  • The EJB webservice implements the asynch interface and acts as a client to the callback interface.

The development is quite simple. The only thing that needs special attention is checking the endpoints used for the EJB WS and the callback interface.

  • When the EJB WS is created using "Webservice from WSDL" the tooling modifies the WSDL supplied endpoint URL. So the wsdl used in the bpel project needs to be updated accordingly.
  • The callback WSDL generated by the Open ESB tooling includes an Application Variable to represent the URL port to use. The regular EJB JAX-WS tooling doesn´t understand Application Variables, so the port needs to be set in the WSDL before the client code is generated.
Screencast (approx 15mins)

Tuesday Mar 17, 2009

Constraints on BPEL Correlation in Open ESB

One of the useful features of BPEL as an orchestration language is its built-in support for correlation – the ability to correlate callbacks from asynchronous services with the original invoking process instance. If you´d like an intro to Correlation, the team at Pymma have written a nice tutorial explaining the concepts of BPEL Correlation and how to make use of it in OpenESB.

Correlation works well when BPEL is used as a simple scripting language for basic webservices, but there are things to consider when using it to orchestrate services offered in a JBI-based infrastructure or when combining correlation with webservice standards such as reliable messaging or transactions. This article discusses constraints on BPEL Correlation in the JBI-based infrastructure of OpenESB, GlassfishESB, and JavaCAPS.

The basic Correlation scenario is quite simple:

  1. the process invokes an asynchronous service with an in-only message (i.e., no reply)

  2. the process continues to execute until the service sends a callback response

  3. the process engine receives the response and uses the correlation information to match the response to the original invoking process instance

  4. the process instance receives the callback msg and processes it accordingly

And this is exactly what happens if you implement a bpel process which calls an over-the-wire WS-I Basic Profile compliant webservice using OpenESB.

However, if you include the Asynchronous Service in the same Service Assembly (aka Composite Application) as the BPEL process, then the process fails to continue executing. In fact, the entire process waits endlessly. To understand why this occurs you need to look a bit deeper at what is happening under the covers in the JBI infrastructure.

Here´s how the basic Correlation scenario works again, but this time including more detail about what is happening under the covers. To follow this you should have a basic understanding of JBI concepts. This page and presentation screencast are useful places to start.

  1. the process invokes an asynch service with an in-only message (i.e., no reply)

  • the BPEL Service Engine (SE) wraps the service invocation in a Normalized Message.

  • the Normalized Message (NM) is wrapped in a Message Exchange (ME) object. In this case, it is a “in-only” Message Exchange.

  • the Message Exchange is sent from the BPEL SE to the Normalized Message Router (NMR).

  • the NMR determines that the requested service is offered by the SOAP Binding Component (BC) and routes the NM to that component.

  • the SOAP BC returns an Acknowledgement (ACK) to the NMR

  • the NMR passes the ACK back to the BPEL Engine

  • the BPEL Engine allows the process instance to continue

  1. the process instance continues to execute until the remote service sends a callback response

  • the SOAP BC creates a Normalized Message which is passed back to the BPEL SE with a similar set of steps to those described previously.

  1. the BPEL Engine receives the callback response and uses the correlation information to determine which process instance the response should passed to.

  2. the process instance receives the callback msg and executes accordingly

This more detailed look at what is happening in the JBI infrastructure seems like a lot of overhead but remember that its only Java objects communicating in the same JVM. Performance benchmarks have shown that there is very little overhead for this level of indirection.

So, why does the scenario fail when the Asynch Service is included in the same Composite Application rather than being invoked as a remote webservice? The issue concerns Systemic Qualities which are the non-functional attributes of the JBI Components offered by GlassfishESB.

Systemic Qualities provide for a number of capabilities which allow GlassfishESB to guarantee different Quality of Service delivery types. E.g., once-and-only-once, at-most-once, and at-least-once. For instance, the Recoverability Quality guarantees that Service Engines can recover from a failure without loss of message. To achieve this when two Service Engines are communicating, the provider Service Engine (e.g., the JavaEE SE providing the Asynch Service) will not send the ACK message to the consuming Service Engine (e.g., the BPEL SE) until the EJB has completed processing. That way, if a system failure occurs during the execution of the Asynch Service, it could be reprocessed without violating the one-and-only-once delivery type.

The consequence of this behaviour, i.e., not sending the ACK until the EJB completes, is that the Business Process does not continue executing. The BPEL SE must wait until the ACK is received from the JavaEE SE. Similarly, when the callback msg is sent, the JavaEE SE must wait to receive the ACK from the BPEL SE. The system then goes into an endless wait-state. The Asynch Service tries send the Callback msg to the process, but the JavaEE SE will never receive the ACK that its callback msg has been received by the BPEL SE because the BPEL SE is still waiting for the ACK for the original request.

This situation occurs for any correlation scenario where the Asynch Service´s container (i.e., the Service Engine in which it executes) provides an ACK to the BPEL SE _after_ the Asynch Service satisfies the request (ie., completes) rather than on receipt of the request. In the original scenario, i.e., when the BPEL process invokes an external webservice, the ACK is sent back from the SOAP BC to the BPEL engine on receipt of the service invocation. However, if the SOAP BC was configured to use a standard that required a different Quality of Service, e.g., WS-ReliableMessaging, then the SOAP BC would only send the ACK after receiving an ACK from the remote service. Similarly, if the BPEL process was configured to utilise transactions (i.e., atomic process) then the ACK would not be received upon initial receipt.

This constraint on BPEL Correlation in Glassfish ESB is not as serious a limitation as it may seem. The use of asynchronous communication is usually used in the implementation of long-running business processes where the importance of these Quality of Service features have a different weighting in your analysis of alternative solutions when compared to fast-running processes. Moreover, other design approaches to implement “correlation” can be achieved without relying on the OOTB functionality of the BPEL Engine.

The support for OOTB Correlation is one reason why BPEL can be sometimes be a useful technology for service orchestration. But blindly relying on technology features is never a substitute for good design.

Friday Mar 06, 2009

Glassfish ESB Systemic Qualities

One of the most frequently asked questions about Open ESB is, "What´s the difference between Open ESB and Glassfish ESB?".

Glassfish ESB is a subset of Open ESB and contains only those JBI components that Sun believes are "stable and ready for production use".  One of the characteristics that distinguishes these Glassfish ESB components and other Open ESB components is the implementation of Systemic Qualities.

This wiki page describes the Systemic Qualities implemented for Glassfish ESB components. These components are also available as part of Java CAPS.

Tuesday Mar 03, 2009

Using encoders to parse CSV files in OpenESB

This screencast provides a simple example of encoding a multi-line CSV file into XML as part of a Glassfish ESB application.

The JBI framework underlying Open ESB, Glassfish ESB, and Java CAPS requires XML msgs to move between the service units. Custom Encoders provide a means to encode/decode structured data to/from XSD defined documents. Encoders are then applied to payloads as the pass through Binding Components on their way in/out of the framework.

This example was plagiarised reused from an example by Sebastien. It shows a multi-line CSV file generated from an OpenOffice spreadsheet. A custom encoder is then created to translate it into a well-defined XML file when it picked up by the File BC. The encoder is enhanced to remove quote marks that are added by OpenOffice to string values when saving as CSV.

The Open ESB wiki has more information about Encoders and examples.

Screencast (11mins)

Friday Feb 06, 2009

Routing with EJBs in OpenESB

A common myth about OpenESB is that you can only route msgs using a full orchestration language such as BPEL.

It is possible to route msgs using a number of mechanisms in OpenESB. These include:

This article looks at how to route msgs between two EJBs that are deployed to the JavaEE Service Engine. I picked JavaEE SE because it is a production quality, supported component as part of the GlassfishESB release of OpenESB. Hopefully POJO SE will also be a supported option soon.

The JavaEE Service Engine (JavaEE SE) is used to allow EJBs to act as Service Units in a JBI-compliant infrastructure. For this example, there are 2 important points to remember about JavaEE SE and JBI.

  • Service Units must be described using abstract WSDL definitions
  • The JavaEE provides a "wrapper" or "bridge". It allows EJBs which are implemented as WebService providers and consumers to act as JBI Service Units. The important point is that although you implement JavaEE SUs as WebServices, they do not necessarily send msgs as SOAP over HTTP requests over-the-wire. If an EJB is acting as a JBI Service Unit, then JaveEE SE intercepts the call and passes the msg internally. This article provides more detail on how JavaEE SE does this. 
This article creates a simple example with two EJBs. One is a simple HelloWorld Service. The other is a RouterService which accepts requests and passes them to the HelloWorld Service.

There are two primary ways to create an EJB WebService.

  • Create a WSDL file, the generate a Webservice from WSDL.
  • Create the EJB and use Webservice annotations to expose it as a webservice

Because all JBI Service Units must be described using a WSDL description, it is easiest to use the "Webservice from WSDL" approach as that will make sure a WSDL exists for our SUs. It is also possible route between EJB Service Units that are created by adding webservice annotations to the EJB, but the approach is slightly different. I might show how to do that in a subsequent article.

The screencast is recorded using Jing, which has a 5min limit on videos, so its split into 4 parts.

Part 1

Note: Normally you should only need to create an Abstract WSDL to repsenent the EJB Service. However, in Part 3 we will create a client for this service, and the client generation tooling currently requires a Concrete WSDL to work. So in this example I´ve created a Concrete WSDL for the HelloWorld service. Hopefully this bug in Netbeans will be fixed soon.

Part 2

  • fill in ejb HelloWorld java code
  • create an EJB project for the Router service
  • import the same XSD files to the project
  • create a WSDL file based on those two XSDs. 
  • create new "Webservice from WSDL"
  • this creates a service stub
  • Screencast

Part 3

  • edit the ejb Routing java code.
  • create a "Webservice Client" and use the wsdl for the ejb HelloWorld service.
  • this creates a Webservice reference. Drop the webservice reference into the Routing code.
  • edit the java to copy the parameter from the Routing request to the HelloWorld request
  • copy the return values
  • create a Composite Application project (JBI Service Assembly)
  • drag´n´drop both EJB projects into the compapp project. Build.
  • add a SOAP connector to act as an external binding for the ejb Routing Service.
  • Screencast
Note: you will notice that the Routing SU consumes an endpoint called javaee_HelloWorld_Port and the HelloWorld SU provides an endpoint called javaee_HelloWorld_Port. Although these two endpoints are not connected in the CASA diagram, the msg will pass between them.

Part 4

  • before deploying, disable the soap binding that connects to the HelloWorld service. This SOAP binding was included because we needed a concrete WSDL for the workaround discussed in Part 1. Disabling the SOAP endpoint is discussed in more detail in this article
  • deploy.
  • create a unit test
  • test. The test msg is sent to the Router. It will then send the msg on to the HelloWorld service which returns "Hello " + requestMsg
  • Screencast

Wednesday Feb 04, 2009

OpenESB Training in Oslo

Later this month (Feb 16-19) I´ll be providing a 3 day OpenESB / GlassfishESB training in collaboration with one of our partners - FreeCode.
(here´s a google translation of the course description)

It will be in English, so if you´re in the Scando area and want a bootcamp introduction to OpenESB / GlassfishESB, please sign up and come along.


Day 1:
- Overview of GlassfishESB, OpenESB, JavaCAPS
-  Architecture overview of JBI technology
- break
- Hands-on Lab: JBI Concepts Pt1
-  lunch
-  BPM support and Systemic Qualities
-  break
-  Hands-on Lab: JBI Concepts Pt2

Day 2:
-  Routing mechanisms BPEL, Java, Camel, & IFL
-  WebService Interoperability Techology (WSIT)
- break
-  Hands-on Lab: WS-Security using WSIT
- lunch
- Encoders for non-XML data
- Hands-on Lab: encoders
-  break
-  Application Configuration
-  Hands-on Lab: Application Configuration

Day 3:
- Management and Monitoring
- Hands-on Lab: Management and Monitoring
- break
- Intelligent Event Processor
- Hands-on Lab: Intelligent Event Processor
- break
- Hands-on Lab: Intelligent Event Processor (cont)
-  Fuji: Technology for GlasfishESB v3
- break
- Hands-on Lab: Fuji

Thursday Dec 18, 2008

Application Configuration for JBI Applications

Application Configuration for JBI components

This how-to shows how to use the configuration capabilities of the platform to modify connection parameters of JBI components without requiring applications to be redeployed. The technique is applicable to JavaCAPS, GlassfishESB, and OpenESB.

The scenario is based on Hemant's Application Configuration Lab How-To which implements a simple routing to read an input file and persist the data in a Database table using the Repository-based technology of JavaCAPS. This example implements the same scenario using the JBI technology.

The screencast consist of the following parts:

  1. Creating a simple routing using a File BC and a Database BC
  2. Configure the input filename using an Application Variable
  3. Configure the output database connection using an Application Configuration Object .

You can see some background information about Application Configuration capabilities in this narrated presentation.


The how-to assumes that you have already created two Databases - test and prod - and the associated Application Server JDBC resources. Part 5 of the GlassfishESB Introduction Tutorial shows how to create a database and setup the appropriate JDBC resources in Glassfish. This example uses JavaDB for the database, whereas how-to for the Repository technology used a MySQL database.

To create the database tables you can use a modified version of the same script. However to change from MySQL to JavaDB you will need to change the datatype LONG to BIGINT.


You can then create the required AppServer JDBC resources and connection pools from the cmd-line

Here's the commands to create the resources for the Prod database. 

> asadmin.bat create-jdbc-connection-pool --datasourceclassname org.apache.derby.jdbc.ClientDataSource --restype javax.sql.DataSource --property password=admin:user=admin:databaseName=prod:serverName=localhost:portNumber=1527:loginTimeout=0 ProdDBPool
> asadmin.bat ping-connection-pool ProdDBPool
> asadmin.bat create-jdbc-resource --connectionpoolid ProdDBPool jdbc/ProdDBPool

Configuration Mechanisms

The platform provides two mechanisms for modifying connection properties:

  1. Application Variables: define variables in a binding component service unit and provide values for those variables in the Binding itself.
  2. Application Configuration Objects: a predefined set of properties for a Binding. Define a instance of this object in the Binding itself and set the values for the predefined properties. Then configure a binding component service unit to make use of this particular named instance
    E.g., here's how Application Configuration Objects and Variables work for HTTP BC

In both alternatives, the composite applications you build can reuse named Application Variables and Application Configuration Objects.

Its not easy to see why there needs to be two mechanisms for doing this, but as a general rule of thumb, you can check if a Binding supports the property you need to configure in its Application Configuration Object. If not, then you can try an Application Variable instead.

Configuring the Scenario

The first configuration we need to make is to change the expected input filename. You can see from the File BC Application Configuration Object that its not possible to do this with an ACO. So this how-to will use an Application Variable instead.

The second configuration is to change the database connection from the Test DB to the Prod DB. This config is done by modifying the JNDI name of the AppServer JDBC Resource that our Database BC Service Unit uses.

In both cases, when the composite application is modified to make use of the Application Variable or Application Configuration Object then it needs to be redeployed. However, once it is setup correctly, then both the Application Variable or Application Configuration Object can be modified by independantly of the composite application. The composite app only needs to be restarted - not redeployed - for the new configuration information to take affect.


Here's the screencast of the scenario

(The blog entry is also available as an article on the Grok JavaCAPS wiki)

Tuesday Dec 09, 2008

GlassfishESB Introduction Tutorial

With the release of GlassfishESB just around the corner, I've updated the original OpenESB Introduction Tutorial based on the new tooling.

The GlassfishESB Introduction Tutorial also adds a fifth installment which shows how to persist data to a database using the Database Binding Component.

  • the process orchestration is implemented in BPEL
  • the process is exposed to Applicants as a WS-I Basic Profile compliant webservice
  • the process invokes a JavaEE EJB without making an over-the-wire webservice call.
  • after processing, the process invokes a ReportMail Service which is exposed via JMS
  • the Loan Application Process is then invoked through the file systems (e.g., incoming ftp msg) in addition to the webservice endpoint.
  • finally, the tutorial shows how BPEL can be used to persist data in a database - for those times when it makes sense to expose database operations at the service level.

If you would like to understand more about what is happening under the covers you can also check out the Introduction Screencast which manually walks through the example and shows how the Java Business Integration run-time provides the loose coupling needed to make everything work together.

Finally, don't forget Mike's introduction screencast which provides a gentle introduction to GlassfishESB.

Monday Dec 08, 2008

JavaCAPS Users Conference - WrapUp

(I meant to post something a few weeks ago but times have been a bit busy lately)

The European JavaCAPS users conference (Horizons) went off better than expected. We ended up with about 200 attendees, and had to turn people away due to lack of space.

The PDF's for the presentations are available on the conference agenda page. Additionally, 12 hands-on labs for CAPS6 were developed. These are available on the JavaCAPS wiki. Some need to be cleaned-up to be used generically and I hope to get around to it soon.

The focus this year was the new technology in JavaCAPS 6 so next year we can concentrate more on customer experiences, best practices, and technical solutions. The unconference and engineering feedback sessions proved popular so we'll look to improve how we incorporate content from customers and partners in next years event.

Thanks to all who attended and helped out. Hopefully next year can be even more successful.

Wednesday Aug 27, 2008

OpenESB in Wonderland

A large part of my job is spent travelling around Europe doing customer workshops on OpenESB and JavaCAPS. While its interesting to see so many different projects, it also results in a lot of dead time sitting in planes, taxis, and hotels. As Tim Bray notes, sometimes the Work/Travel Ratio isn't good for anyone. So I started looking at Project Wonderland as a way to do things remotely, when it makes sense to do so, but with more interaction than remote desktops and tools like WebEx

If you haven't heard about Wonderland, you can imagine it as a private version of SecondLife but with lots of additional features for workplace collaboration. For instance:

  • directional audio and softphones for communication between avatars
  • virtual conference phones for people to dial in and out to the real word
  • video conference abilities using external cameras
  • in-world Video and PDF Viewer application
  • a Cone Of Silence for private conversations (and a bonus for anyone who grew up watching Get Smart)
  • and, most importantly for our goal, the ability to run and share applications between avatars in the world

As a proof-of-concept I modified the default virtual world that comes with Wonderland to provide a virtual workshop environment for OpenESB and JavaCAPS. Then I invited some other Sun guys in the field to come and test it out. Sebastien, Louis, Mark, and myself are distributed throughout Europe but we were all able to join the server and share the OpenESB design-time and run-time environments to simulate a virtual workshop environment.

View Larger Map

Check out our poorly acted example workshop below. Note: the camtasia recording seems to have made the audio a bit crackly in places. The live audio is much better than what is reproduced in the screencast

As a proof-of-concept it worked quite well. Now we can trial it with some guinea pigs customers, perhaps use it as a way of supporting more attendees at the JavaCAPS Horizons event, or maybe even make pre-packaged workshop environments for customers with distributed development teams.




« April 2014