"When Connection.close() should not close", or the J2EE JCA ManagedConnection life cycle

One of the things I've struggled with most when I was developing the JMSJCA Resource Adapter at SeeBeyond, was the life cycle of the ManagedConnection. I wasted numerous hours going into fruitless directions simply because I did not fully grasp the intricacies of the ManagedConnection life cycle. If you're involved in developing Resource Adapters, you're likely to stumble onto the same problems, so read on!

What do you mean, close() should not close?

Let's take a look at how you might use a JMS Connection in an EJB:

@Resource(name="jms/cf") ConnectionFactory cf;
@Resource(name = "jmx/q1") Queue q;

@TransactionAttribute(TransactionAttributeType.REQUIRED)
public void myBusinessMethod() throws Exception {
Connection c = cf.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE);
s.createProducer(q).send(s.createTextMessage("Hello world"));
c.close();
}

Should c.close() really close the connection?

Keep in mind that since J2EE 1.4, JMS providers interface with the application server using JCA. JMS connections should be pooled so that repeated execution of the statement fact.createConnection() is cheap. So the answer is no, the connection should not really be closed.

Should c.close() return the connection to the pool then, so that another EJB could use it? Remember that there is a transaction in progress, so the container should hold on to the connection until the transaction is completed. Only then should the container return the connection to the pool.

Are you appreciating yet the complexities that the application server has to deal with? If not, let's add the following to the method above:

@TransactionAttribute(TransactionAttributeType.REQUIRED)
public void myBusinessMethod() throws Exception {
ConnectionFactory fact = (ConnectionFactory) new InitialContext().lookup("jms/cf");
Connection c = fact.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE);
s.createProducer(s.createQueue("q")).send(s.createTextMessage("Hello world"));
c.close();

otherMethod();
}


public void otherMethod() throws Exception {
ConnectionFactory fact = (ConnectionFactory) new InitialContext().lookup("jms/cf");
Connection c = fact.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE);
s.createProducer(s.createQueue("q")).send(s.createTextMessage("Goodbye world"));
c.close();
}

When myBusinessMethod() is called, which in turn calls otherMethod(), how many connections are created? Good application servers like the Java CAPS Integration Server, the Sun Java System Application Server, and Glassfish use only one connection in this example. The connection that is used in otherMethod() is the same connection that was created in myBusinessMethod().

Let's look in more detail at what is happening under the covers, and why that is important.

Alphabet soup

Before we begin, let me reiterate some of the terms and abbreviations used. JCA stands for the Java Connector Architecture. It provides the interfacing between applications running in an application server, and external systems such as CRM packages, but also systems such as JMS.

A Resource Adapter is a set of classes that implement the JCA. The central interface for outbound communications is the ManagedConnection. An application, e.g. an EJB, never gets access to a a ManagedConnection directly. Instead, it gets a connection handle. The handle in the above example is the JMS Connection (actually, it is the JMS Session -- but let's not go into that right now). This Connection object is not the JMS Connection that is implemented by the JMS provider, but is a wrapper around such a connection. The wrapper is implemented by the Resource Adapter.

A ManagedConnection holds the physical connection to the external system, e.g. the JMS Connection and Session. Because the physical connection and hence the ManagedConnection is expensive to create, the application server tries to pool the ManagedConnections.

State transitions, and why they are important

We've already seen that a ManagedConnection can be in an idle state or pooled state and it can be in use. It would be very nice if the application server would tell the ManagedConnection when these state transitions happen. But it doesn't. Why would the ManagedConnection care to know when it is being used or when it is being returned to the pool? Let's look at an example in JMS. Let's change the example a little bit:

@TransactionAttribute(TransactionAttributeType.REQUIRED)
public void usingTemp() throws Exception {
ConnectionFactory fact = (ConnectionFactory) new InitialContext().lookup("jms/cf");
Connection c = fact.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue temp = s.createTemporaryQueue();
Message request = s.createTextMessage("541897-9841");
request.setJMSReplyTo(temp);
sendRequest(request);
Message reply = s.createConsumer(temp).receive();
c.close();
}

The sendRequest() method would somehow start a new transaction and send the request message. The question is when the temporary destination should be deleted. According to the JMS spec, the temporary destination should be invalidated as soon as the connection is closed. However, a transaction is still in progress, so the JMS provider will throw an exception if the temporary destination is deleted when c.close() is called. Can't we just ignore the temporary destination? After all, it will be deleted some time, e.g. when the physical JMS connection is closed. However, since the connection is pooled, it may take a long time before the physical JMS connection is closed. During that time temporary destinations just keep piling up. This approach will exhaust the JMS server.

The temporary destination can be deleted safely when the ManagedConnection is returned to the pool. And that's why it's important to know the state transitions.

How to detect state transitions

What hints does the application server give the ManagedConnection about the state transitions? Let's take a look at the ManagedConnection interface:

public interface ManagedConnection {
Object getConnection(Subject subject, ConnectionRequestInfo connectionRequestInfo) throws ResourceException;
void destroy() throws ResourceException;
void cleanup() throws ResourceException;
void associateConnection(Object object) throws ResourceException;
void addConnectionEventListener(ConnectionEventListener connectionEventListener);
void removeConnectionEventListener(ConnectionEventListener connectionEventListener);
XAResource getXAResource() throws ResourceException;
LocalTransaction getLocalTransaction() throws ResourceException;
ManagedConnectionMetaData getMetaData() throws ResourceException;
void setLogWriter(PrintWriter printWriter) throws ResourceException;
PrintWriter getLogWriter() throws ResourceException;
}

The getConnection() method tells the ManagedConnection to create a new connection handle, so after that method the ManagedConnection knows that it is not in the pooled state.

The destroy() method destroys the ManagedConnection so it signals a state transition to "non-existent".

Doesn't the cleanup() method indicate that a connection is no longer used and will be returned to the pool? It depends on the application server when exactly this method is called. Most application servers will call cleanup() immediately when the application calls Connection.close(). At that moment the connection may still be enlisted in a transaction, and may be reused as we've seen in the examples above.

As it turns out, there are two states that the ManagedConnection needs to keep track of so that it can detect a state transition from "in-use" to "pooled".  The two diagrams keep track of whether the application has access to the ManagedConnection through a connection handle. In other words, it keeps track of whether the ManagedConnection has any outstanding connection handles. See the figure below.

The second state is a transactional state: it keeps track of whether the ManagedConnection is enlisted in a transaction. See the figure below:

Let's look at a few examples that illustrate this concept:

@Resource EJBContext ctx;
@Resource(name="jms/cf") ConnectionFactory cf;
@Resource(name = "jmx/q1") Queue q;

public void ex1() {
ctx.getUserTransaction().begin();
Connection c = cf.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE);
s.createProducer(q).send(s.createTextMessage("Hello world"));
c.close();
ctx.getUserTransaction().commit();
}

In this example the getConnection() method is called upon cf.createConnection().createSession(): the ManagedConnection is in use. At the same time the connection is enlisted in the transaction; this is done through the XAResource obtained through ManagedConnection.getXAResource(). When c.close() is called, the ManagedConnection is no longer accessible to the application: all the connection handles are closed. The application server may and probably will call ManagedConnection.cleanup(). However, the connection is still enlisted in the transaction, so the connection is not returned to the pool yet. That happens when getUserTransaction().commit() is called. While the connection is still enlisted, the resource adapter should not try to delete any objects such as temporary destinations in the example mentioned above.

In the following example the enlistment happens a little later. See the inline comments for the state transitions.

public void ex2() {
Connection c = cf.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE); // Accessible, but not enlisted
ctx.getUserTransaction().begin(); // Accessible and enlisted
s.createProducer(q).send(s.createTextMessage("Hello world"));
c.close(); // Inaccessible and enlisted
ctx.getUserTransaction().commit(); // Inaccessible and not enlisted
// Return to pool
}

There is also a possible transition from "Inaccessible and enlisted" back to "Accessible and enlisted":

public void ex3() {
Connection c = cf.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE); // Accessible, but not enlisted
ctx.getUserTransaction().begin(); // Accessible and enlisted
s.createProducer(q).send(s.createTextMessage("Hello world"));
c.close(); // Inaccessible and enlisted

c = cf.createConnection();
s = c.createSession(false, Session.AUTO_ACKNOWLEDGE); // Accessible and enlisted
s.createProducer(q).send(s.createTextMessage("Hello world"));
c.close(); // Inaccessible and enlisted
ctx.getUserTransaction().commit(); // Inaccessible and not enlisted

// Return to pool
}

The following example shows that there can be multiple connection handles:

public void ex4() {
Connection c = cf.createConnection();
Session s = c.createSession(false, Session.AUTO_ACKNOWLEDGE); // Accessible, but not enlisted
ctx.getUserTransaction().begin(); // Accessible and enlisted
s.createProducer(q).send(s.createTextMessage("Hello world"));

Connection c2 = cf.createConnection();
Session s2 = c.createSession(false, Session.AUTO_ACKNOWLEDGE);// Accessible and enlisted
s2.createProducer(q).send(s2.createTextMessage("Hello world"));
c2.close(); // Accessible and enlisted
ctx.getUserTransaction().commit(); // Accessible and not enlisted
c.close(); // Inaccessible and not enlisted
// Return to pool
}

How to monitor transaction enlistment

To monitor enlistment and the commit() or rollback() method on the XAResource it is possible to write a wrapper around the XAResource of the underlying provider. There are some drawbacks associated with that, and it is better to monitor the transaction through a javax.transaction.Synchronization object. I've described this in more detail in my blog of July 23, 2006: J2EE JCA Resource Adapters: The problem with XAResource wrappers .

Conclusion

By keeping track of both the number of outstanding connection handles given out to the application and the enlistment in a transaction, the ManagedConnection can figure out when it is returned to the pool so that it can undertake necessary actions such as destruction of objects indirectly created by the application.

Comments:

Frank, I learned something new again with your example about myBusinessMethod() and otherMethod(), thanks ! I just think you should have elaborated a little bit more on how the connection gets reused as it requires close interaction between the connection/session pool and the TM. Anyway, I will manage to implement it and solve the quick pool depletion problem I was running into. You also forgot to mention that this is not always the case that connections/sessions must be kept out of the queue until after commit: some resources support transaction interleaving thus allowing the connection to be released as soon as close is called.

Posted by Ludovic Orban on August 27, 2006 at 06:00 PM PDT #

Hi Ludovic, Thanks for your feedback! > should have elaborated a little bit more on how the connection gets reused as it requires close interaction between the connection/session pool and the TM. Are you referring to the use of the ConnectionRequestInfo object etc? > some resources support transaction interleaving thus allowing the connection to be released as soon as close is called. Are you referring to the mechanism of LazyEnlistableConnectionManager? Or am I missing something? How would the appserver recognize such a resource? And how would the appserver know that the application would no longer need to re-use the same connection? BTW, I looked at your site at http://www.bitronix.be/Btm/Overview. Interesting project! Do you intend to have a persistent transaction log? Or do you intend to do transaction logging (i.e. for recoverability) through some other means? There are some interesting ideas I would like to explore if I had the time. Also at SeeBeyond we developed a mechanism for recoverability of transactions without logging to an external file. I intend to blog about that in a future entry.

Posted by Frank Kieviet on August 29, 2006 at 02:45 PM PDT #

Hi Frank, I'd be happy to discuss this subject with you in more details. Please drop me an email (if you're interrested) at ludovic.orban _AT_ gmail.com as it's more convenient than posting comments. Thanks for your feedback !

Posted by Ludovic Orban on September 01, 2006 at 07:42 PM PDT #

Hi Frank,

you wrote "(actually, it is the JMS Session -- but let's not go into that right now)". This is exactly where some of my problems grasping JMS/JCA lie. As the XASession normaly is the object you get the XARessource from, I'd expect JMS sessions to be the objects in the pool, but all examples go like createConnection().createSession(..), which implies that the connections are the pooled "physical" objects.

Another thing is that connection.close() is (as of API spec) supposed to close the session, too. And the contract of JMS Session requires that any outstanding work must be rolled back when closing transacted sessions. Why is JCA allowed to "break" this contract, even if it surely is "the right thing to do" when using pooling? I am missing some statement in the spec like "not rolling back is allowed (if not required!) in the RA case" or ".. in the JTA/EJB case".

N.B. Your Blog and RA insights in particular are very valuable, if not frustrating :-) , as you solve some problems, which I did not even have before (or realized if have). The only RA experience I have is from implementing an FTP client as outbound RA under JCA 1.0. Now I am planing to encapsulate our JMS implementation as RA in JCA 1.5 and many new questions arise, as the FTP RA was non-transacted :-)

Posted by Michael Bartmann on October 26, 2006 at 02:24 AM PDT #

Hi Michael, Here's what happens:
1 QueueConnectionFactory f = ctx.lookup(...);
2 QueueConnection c = f.createQueueConnection();
3 QueueSession s = c.createQueueSession();

The factory in 1 is an object provided by the RA. The connection in 2 is an object provided by the RA that acts like a factory object for sessions; it does not represent a physical jms connection/session at this point.

When 3 is executed, the connection object from 2 calls into the connection manager and gets a session-connection pair from the pool. The connection and session in this pair are "physical" jms objects. The session object that is returned to the caller (variable s in 3), is a wrapper provided by the RA.

Now let's close the connection:
4 c.close()
Recall that the variable c is a wrapper provided by the RA. It will return the real jms connection/session pair back to the pool and it will mark the session that the application sees (variable s) as closed, so that all operations that the application performs on s will result in an exception.

What will happen to outstanding work when the application calls close()? This is where the behavior is different from a standalone jms client from application code in an appserver. In an appserver the container has control over the transaction; unless the transaction is marked for rollback, it will not rollback the messages in JMS, but commit them.

The reason for the difference in behavior is the following: when doing using JMS in CMT, the connection has got to be closed within the applicaiton code (= within the transaction) to avoid connection leaks. If that would imply that the data in JMS would be rolled back, that would mean that you simply could never use JMS because data would never be committed.

I think you can find this behavior mandated in the EJB spec or J2EE spec (it's indeed not in the JMS spec).

Implementing a spec-compliant RA for JMS is an expensive undertaking, and I recommend to reuse open source as much as possible. The JMSJCA implementation that I have worked on will be open sourced soon; perhaps you could reuse it. JMSJCA has a number of abstractions that makes it especially easy to deal with differences in JMS implementations.

Thank you for your feedback on this blog. Plz let me know how I can improve it.

Also, if I can further clarify things, let me know; I'll be happy to lend a helping hand.

Posted by Frank Kieviet on October 26, 2006 at 03:03 AM PDT #

Hi Frank,

and thank you very much for your comprehensive and unbelievably fast answer.

I must admit that the fact that the appserver (read: JCA pool) provides you with objects which do not exactly behave as their api doc promisses still bugs me. The fact that these objects are wrappers, which _can_ (technically) change the behaviour of their delegate, does not excuse that the wrappes do _themselvse_ formally implement "QueueSession".

A similar thing kept me wondering for a while: the jca spec lets the jca container enlist its XARessource to the tm. Where does the application know from, whether the factories bound in jndi are providing "auto-enlisting" ressources, or ressources which have to be enlistes "manually". The behaviour of a ressource should not depend upon the fact that is is a jca/pooled ressource. So the jca spec might regulate the internal contracts between managed ressources and the jca container, but not the relation between application (session bean) and ejb container/jta.

Perhaps I am missing something here, or I do not understand the "whole picture". Or am I too nitpicking about the role of specs and apis in generall?

Thanks again, Michael

Posted by Michael Bartmann on October 26, 2006 at 11:57 PM PDT #

Hi Michael,

Let me add something to my previous comments: the big difference between a JMS client running in an application server and stand-alone is that the parameters to createQueueSession(transacted, ackmode) are ignored; the behavior is auto-acknowledge from the user's point of view. This "picture" falls apart when you're trying to do something like request/reply (see my previous blog). I'm not sure if this comment clarifies anything or makes it more confusing.

Back to your comments... how does the application know if a resource in JNDI is automatically enlisted in the transaction?  The answer is that doesn't. The intent of as laid out in the J2EE spec is that all resources are transactional and hence automatically enlisted in the transaction. In reality, application servers allow you to create resources such as JMS that are not enlisted in the transaction automatically. In that case the developer and the deployer (I'm using these terms as laid out in the spec) need to agree what is registered in JNDI and what their properties are.

Or am I too nitpicking... no, I don't think so. It's kind of confusing in the beginning, especially going from J2EE 1.3 to J2EE 1.4. I remember it took me quite some time for me to figure out how things were supposed to work when we went to J2EE 1.4. In the end it all made sense to me, but I do remember my initial confusion and frustration, so I know what you're going through.

Michael, out of curiosity, if you don't mind: what kind of resource adapter are you working on?

Kind regards,

Frank Kieviet

Posted by Frank Kieviet on October 28, 2006 at 04:50 PM PDT #

Hi Frank,

thanks for your clarifications, things begin to make sense for me now...

To give you some insight about my environment:

My company ("4Production AG" in germany, with a SUN partnership) builds MES (Manufacturing Executing Systems) for the metal industry. So we have all sorts of integration tasks in every rollout of our (closed source) framework.

For all asyncronous integrations we use our own message service "ms4p". It origins back in 2001, is implemented by an mbean-based stack and has a transport protocoll abstraction layer similar to jgroups (protocol stack with low level for legacy integration TCP/UDP/FileTransfer etc., remote side is often not java based). It implements XASessions, but back in 2001 there was no JCA 1.5, and w/o an inflow contract we did not use JCA integration at all. So there:

  1. is no pool
  2. application (local slsb) has to enlist()
  3. application cannot close() the jms session (neither does the container, so it depends on the garbage collector :-()
Now I wan't to get rid of these shortcommings, and I am analysing who has to enlist/close etc. That's where my questions come from. BTW: if your JMSJCA adapter would allow integration into a closed source environment (license-wise), I'd be very interested in giving it a try. The least thing our company does for the open source world is giving feedback and sharing experience and bug-fixes.

There are a lot of not JMS based integrations, too. I did mention our FTP client, which is JCA based (but not transacted). There are a lot of interfaces, where we have to set digital signals etc., which will fit into the JCA paradigm, too.

King regards,

Michael Bartmann

Posted by Michael Bartmann on October 28, 2006 at 07:09 PM PDT #

Hi Michael,

Thanks for your reply! This looks interesting. Not that I'm trying to sell you anything, but did you consider using an open source JMS server? The Sun JMS server was released into open source recently (JMQ or Open MQ). The JMS server that is bundled with Java CAPS (called STCMS) may also be open sourced at some point; this server has different performance characteristics compared with JMQ / Open MQ. If and when I'm not sure.

The JMSJCA source will likely be released under the CDDL license.

I'm not sure if you're running your code in an application server. In case you're not, here's something else that you may be interested in: I'll likely add a connection manager to JMSJCA so that it can be used outside of an application server while still providing connection pooling, support for transaction managers, and with a solution for closing connections in the middle of a transaction. Similarly there will be some classes that make it possible to leverage the inbound part of the connector outside of an application server.

Frank 

Posted by Frank Kieviet on October 29, 2006 at 11:53 AM PST #

Hi Frank,

I am not too proud to use code not written by myself :-)

Our product, the "4P Factory Suite" runs as a J2EE application and was developed under JBoss. It uses hibernate for persistence, driven by our own MDA framework. For publish-subscriber messaging (i.e. client notification) we use JBossMQ. Beside different ways to deploy JMX-MBeans we have gotten our framework running under BEA Weblogic and SAP Netweaver, where we needed/wrote our own abstraction layer, to provide a JBoss-like MBean "\*.sar" deployer.

But with integration, thats a different story. Often the other side is a control system, which cannot even run C/C++, let aside Java. Communication is purely asynchronous, and the wire protocoll may differ from project to project, but mostly it is plain old TCP/IP with messages consisting of fixed length fields. In the end our message service has a protocol stack which builds higher abstractons around the TCP/IP communication, and provides a JMS API to the business layer. So conceptually our message service it symetric, each side may have buffers etc. and there is no central server instance. We just cut that in two halves, where the other side is often implemented in non-Jave code.

I am aware, that you can achieve a similar result by using any "normal" message service, and a TCP/IP "bridge" agent. This is effectively what we do anyway, as our kernel has an internal JMS API to the transport layer MBeans.

But aside the communication itself, our own implementation has a deeply integrated message browser with Swing GUI, filtering, resending etc. which works on any J2EE server where we can deploy MBeans. Or perhaps, if we get to use JCA 1.5, any server which supports that.

And last but not least our message service runs 24/7 in highly critical production environments, so we are a bit reluctant to change the architecture in a too revolutionary way.

Concerning your JMSJCA: what does a JMS service need to be "compatible". My question arises, because J2EE server integration to feed MDBs is heterogenous unless you already have JCA 1.5. Some servers use what the JMS 1.0/1.1 API suggests: A ConnectionConsumer with ServerSession etc. Thats similar to how our JBoss integration works since JBoss 2.3. But with Netweaver or WebLogic the J2EE server simply used Session/Receiver/MessageListener to read messages from our JMS layer and stuff them into MBeans. So how about JMSJCA?

Kind Regards,

Michael Bartmann

Posted by guest on October 29, 2006 at 05:30 PM PST #

Hi Michael,

Thanks for your response, and I apologize for my tardy response! I agree you better not change anything if there's no need for it.

About your questions: the interfacing between the RA and the JMS provider is pluggable. For inbound, you could easily write a few classes yourself so that you could hookup a JMS provider that's violating the spec in all sorts of ways. To cope with different implementation details of the JMS servers we support, three ways of inbound are provided: synchronous receive with multiple receivers, a single asynchronous receiver (i.e. receiver.setMessageListener(listener)), and connection consumers (i.e. session.setMessageListener(listener)). For the latter there are a few different ways that the session is enlisted in the transaction (in the onMessage() method and in the run() method).

For outbound the situation is similar: through an object factory that can be overridden for each implementation,  jmsjca asks for different aspects of the JMS provider objects, e.g. the session. Session creation etc can be overridden.

I recently validated JMSJCA on Weblogic 9.1; tests have been done on JBoss as well although the test suite has not been ported to JBoss yet; the same goes for WebSphere.

Frank

Posted by Frank Kieviet on October 31, 2006 at 02:25 PM PST #

Hi Frank,

(this time it's me with a tardy response...)

All this sounds like JMSJCA might fit well to most of our requirements. Is there some planned schedule or timeline to release your JMSJCA as open source?

Kind regards, Michael

Posted by Michael Bartmann on November 22, 2006 at 08:44 PM PST #

Hi Michael,

JMSJCA is being open sourced as part of JBI; the binaries are already available as part of the JMS binding component. I expect that the source will be there before mid January.  Will that work for you?

Frank

Posted by Frank Kieviet on November 27, 2006 at 02:21 AM PST #

Announcement: jmsjca is now available as an open source project on java.net: http://jmsjca.dev.java.net

Posted by Frank Kieviet on January 21, 2007 at 01:57 AM PST #

hi,
I am developing a distributed transactions framework using jboss and spring.I want to access the API's also.so can we have custom commit and rollback in jboss.

Posted by jenny on September 30, 2007 at 09:21 PM PDT #

Re Jenny: sorry, I don't know.

Posted by Frank Kieviet on October 03, 2007 at 10:27 AM PDT #

Hi Frank,
Thank you for your article, it's being really useful. I also checked the presentation at javaone.
I just wanted to make sure: the transitions from enlisted-not enlisted are made by calls to start() and commit()/rollback(), performed by the TM to the XAResource associated with the ManagedConnection?
Thank you
Kind Regards

Posted by Enrique on July 24, 2008 at 02:10 AM PDT #

Re Enrique,

Indeed, the transition from enlisted to not-enlisted can be made by listening on the progress of the transaction. This can be done by listening in on the XAResource, but it can also be done by registering a javax.transaction.Synchronization object.

Frank

Posted by Frank Kieviet on July 27, 2008 at 08:04 AM PDT #

Post a Comment:
Comments are closed for this entry.
About

fkieviet

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today