Clustering in JBI Components 2

What’s in store for JBI?
Each server instance, whether it is standalone, DAS or clustered
will contain a JBI runtime. DAS will contain Facade mbeans that
will be communicating to the cluster instances or standalone
instances based on the target information.

Basic JBI runtime will not have any special capability to handle
clustering. The components on top of it will be aware of EE
clustering. So, it will be the component's responsibility to work
in a clustered manner. For example, a BPEL process doing a
correlation will need the messages to routed to the same instance.
However this will be done in a way specific to the component. For
example, BPEL engine deployment to a cluster will use a group ID to
identify the cluster. This can be the cluster name. 
In case clustered environment once we deploy the
ServiceAssembly(SA), the same will be replicated in all the Sever
Instances in the cluster. This mandates that we need to have a
mechanism of identifying what is the component that is servicing
the request on the cluster. This will boil down to associating the
Node Agent Name with the endpoint that is exposed if there is no
identifying port as in case of HTTP/SOAP BC.
That said we need to thrash what are the scenarios for any
component and the what prompts the need for the load balancer for
the components and then the strategies for handling the same and
look for what Glassfish provides via API for that.
1. Outbound communication from the BC to the external system.
a. In Only MEP: Since the message is destined for the external
system and the WSDL contains the external system information the
message can be spitted out without any issues since the message
comes out of the correlated BPEL instance.
b. In-Out MEP: In this case the external system sends either
ACK/NAK to the SA/SU instance in the cluster. This case needs to be
taken care of. This has a catch in view of HA/ or failover, unless
otherwise we have XA enabling with respect to the EIS we can not
guarantee the failover, but since this scenario ends up in deadlock
we can not guarantee the failover.
2. Inbound communication from the external system into the BC
a. In Only MEP: Since the message comes into the BC and is targeted
at a particular instance (endpoint) of the BC this has to be taken
b. In – Out MEP: Since the message comes into the BC and is
targeted at a particular instance (endpoint) of the BC this has to
be taken care.
In both the above cases as in case of EJB container for having
failover support we mandate that the resource happens to be an
XAResouce and is transactional.
Need for Custom Request Router (Load Balancer): I rather do not want to call this as Load balancer rather I would like to call it a request router, because this appeals to me that this name would suite it most. If we go over the sequence of the events we can arrive at the need. This is what happens when we have the Binding Component is built and the same is deployed in the JBI container in a clustered scenario:
  1. Deploy the Component on DAS
  2. Component gets replicated in all the clustered server instances.
  3. Component when installed will call Bootstrap / Lifecycle and
    Deploy(ServiceUnitManager) in that order and the Deploy will take
    care of the Service Unit's that are referenced by this component.
    The component handles the inbound and outbound message requests
    using either ExecutorService / or normal threads.
  4. When ServiceAssemblies gets deployed on the cluster they also get
    replicated on all the server instances and the service units
    contained there off will register the endpoints whether inbound or
    outbound and start doing work.
  5. While the endpoints get registered there should be a way either for
    the failover/recovery or for the stickiness that is required while
    handling the request, we need to have unique way of identifying the
    endpoint which has previously handled the request. In case of HTTP
    LB the LB has a mechanism of logical-physical mapping which will
    service the request.
  6.  For this reason we can have the
    ClusterName+NodeName+ServerInstanceName as a unique identifier
    added for the current/existing endpoint for the BC. For this before
    registering the endpoint the BC should check whether its cluster
    aware and if yes do the way said above if not register the endpoint
    the normal way.
  7. The JBI runtime has to provide for some hooks from with in the
    AppServer, to the GMS module which will pass on the information
    regarding the cluster and the health of the individual components
    that are deployed on the container. I have discussed these classes
    at the end of this document.
  8. Apart from that the component might need to send out notifications
    regarding its own status on the cluster by NotificationEmitter or
    some other mechanism that is outlined by the GSM.
  9. Component can use one of the strategies that are out lined below
    for the load balancing and failover in the component specific way.
    For example, the SOAP BC will use HTTP load balancer and the JMS BC
    will use inbound JMS load balancing.
  10. From other components perspective, we might require to have a load
    balancer for some BCs as in case of JDBC where in we have large
    data, which is in bound or out bound HL7 picking up from a DB/file
    system and sending out data to the external system. We need to
    implement algorithms based on volume and size of the data that is
    coming in. We might even have to give the option of selecting the
    load balancing algorithm to the user.
Strategies for Binding Components:
1. Using JXTA / JGroups: Both these APIs are good enough for peer
to peer based communication and in fact AppServer has hooks to both
these APIs and uses JXTA as its protocol for GSM. These provide
APIs on the client side for the component to hook to the server.
Both models support virtual synchrony and dynamic finding of the
machines in the cluster etc. For load balancing here we can think
of other strategies based on the request that is coming and the
data that needs to be handled by way of peer to peer communication,
sending the request context from one peer to the other peer if
other has already the requested data.
2. Custom way: we can think of using single database for storing
the request information and client session with respect to each
protocol. The request information includes the key that uniquely
identifies the session for the req. and can carry on the work. This
can be combination of the cluster name, instance name and the
schema + table name in case of JDBC etc. We can think of lead time
here for catering to fail over where in if in the specified time if
the request is not catered to, another instance can set its key
take the request context and can proceed with handling the request.
There is a catch here, if the resource is participating in a
distributed transaction since one TM is already involved in
handling the transaction (and transactions are atomic), we might
need to wait till the Server instance comes up. But here too we can
have a mechanism by way of knowing that the said tuple is under a
transaction and emit a notification for the System Administrator to
take care of the same.
I will be discussing regarding the API support and other implementation details in my next post

Post a Comment:
Comments are closed for this entry.

I was part of Sun R&D in Java CAPS and later Glassfish ESB. I moved from R&D to Consulting. I am currently working as a Solution Architect in Oracle Consulting Services (India). I was sharing my experience w.r.t. Java CAPS and other technologies during Sun period. Now in Oracle world I share my experiences with Oracle FMW product line as well as other Oracle Technologies and products.


« July 2016