The Problem
The problem was they have a number of applications deployed to a cluster as clients consuming from the distributed topic. Each instance of the application is treated as a consumer and thus getting multiple message instead of the desired one per application across the cluster. Now don't get me wrong, the customer was not complaining that JMS or distributed queues needed to be changed, but how could they get the functionality they wanted while still enjoying the higher availability and scalability a UDT afforded them. One thing to note is if the operations being performed on the messages were idempotent this wouldn't have been as serious even though it was processed multiple times, the result would have been the same (albeit a little ugly!).

The image below describes the situation the customer was trying to solve.

The Target
The goal was to have a messaging system to the guarantee delivery while also being resilient. Each application, whilst deployed across the cluster should only get one message.

This diagram shows the proposed messaging architecture.
A message coming into the Uniform Distributed Destination (UDD) is sent, in some load balancing fashion, to only one instance of the application. The load balancing might be multiplexed across application or across cluster nodes and then across the instances of the application on that node.

Proposed Solution

The main problem with the current architecture is the UDT which is directly handling the incoming the messages and from which the applications are directly subscribing to. This naturally broadcasts the messages across all members indiscriminately, irrespective of application.

The UDT can be replaced with a UDQ to which external messages are published. The UDQ will load balance these incoming messages to one and only one of the managed servers. A local Message-driven Bean deployed to the cluster can listen on the UDQ (an instance of which will be deployed to each of the managed servers). Its sole job is to publish to a local-only JMS Topic to which the applications are listening. This is shown logically below.

It is important the MDB is transactional, to ensure it succeeds in putting the message into the local topic, or on failure the transaction is rolled back and thus the message is ready for another attempt. The local Topic must have a unique name and no global JNDI name. Instead a local JNDI name is provided which is the same across the cluster (and thus the MDB publishes the messages to a Topic with the same same name irrespective of its location in the cluster).

I have prototype this solution in WebLogic Server 10gR3 using EJB 3.0 on a two node cluster, and am happy to share the project should someone be interested enough.

Assumptions
This blog entry assumes the following
1. You have a proficiency in WebLogic Server and JMS
2. You know about Java and C coding

Notes
While this blog reflects work done using WebLogic Server 10gR3, it should work with all WebLogic Server version 9 and above (for SAF).

Introduction
The JMS C API uses JNI to access the standard WebLogic JMS Libraries. The API follows the same flow as the Java equivalent (funnily enough) where a context is used to look up the connection factory and destination and the connection factory creates the connection etc etc.

Setting up your Environment
To use the C JMS API you need to set up your environment, this blog assumes Unix/Linux but the same principles apply for Windows users.
1. Set up the CLASSPATH to include, the weblogic.jar file (found in /wlserver_10.3/server/lib - the path will change depending on the version you're using) or you can use the wlsjmsclient.jar instead of the full jar file if you wish.
2. Set up the library path so your executable knows where to find the jvm and jni shared libraries. This equates to setting the LD_LIBRARY_PATH to something like /wlserver_10.3/server/native/linux/i686:/jdk160_05/jre/lib/i386/server/.

The C Code
Once you have done this you're ready to work on the code. The follow code 'skeleton' shows how to publish a message to a JMS Queue.
1. Get the context using JmsContextCreate
2. Use the context to get the connection factory using JmsContextCreateConnectionFactory and the destination (the queue) using JmsDestinationCreate
3. Get the connection to the JMS Server using JmsConnectionFactoryCreateConnection
4. Create the session using JmsConnectionCreateSession
5. Create the producer (or consumer) using JmsSessionCreateProducer (or check me)
6.Create the message (in this case a Text Message) using JmsSessionTestMessageCreate
7. Send the message using JmsProducerSend

When using the C API it is critical all handles returned from the API functions are closed or destroyed correctly using the appropriate -Destroy or -Close functions, failing to do so could cause memory leaks (don't depend on GC, especially when using the C API).

Compiling
When compiling you need to make sure you
1. Include the WebLogic C API header files (located in /wlserver_10.3/server/include)
2. Link the libjmsc.so, libjvm.so, and libpthread.so with your code.

Here is an example command (using gcc):
gcc -I/usr/local/bea/WLS103/wlserver_10.3/server/include -L/usr/local/bea/WLS103/wlserver_10.3/server/native/linux/i686/ -L/usr/local/bea/WLS103/jdk160_05/jre/lib/i386/server -ljmsc -ljvm -lpthread -o JMSClient JMSClient.c

You could use make to automate a lot of this.

Running the Binary
Run the binary as you would any other linux/unix binary, while making sure the library path and classpath is set correctly

Configuring the Store-and-Forward
Once you have the simple producer working and publishing messages to a queue or topic you are ready to setup the SAF.

You need to export the SAF Context file as per the SAF documentation. Once you have done this you need to make sure you use weblogic.jms.safclient.jndi.InitialContextFactoryImpl as the JNDI provider (passed as a parameter to the JmsContextCreate function and the URI (or connection string) needs to be the location of the SAF Context you exported. Make sure you have either weblogic.jar or wlssafclient.jar in the classpath.

Because the SAF Sending Agent runs in a separate thread to your main thread there can be some latency between your producing the first message and it getting to the destination. This thread also means you need to make sure you don't exit the program too quickly, otherwise nothing will be sent (you keep exiting before the SAF Sending Agent can initialise)!

Download Example

You can download an example I have created here, this is only meant as an example and there is no warranty, use it at your own risk.

References
1. WebLogic JMS C Documentation
  a) JMS C Documentation
  b) The WebLogic Server 10gR3 JMS C API
2. WebLogic Store-and-Forward Client Documentation