Making your JBI Component Cluster aware in Glassfish
By narayanaa on Dec 06, 2006
The approach I took while writing this note is to give an over view of clustering, the components that are involved in making the clustering possible and also discuss the strategies that AppServer took while making its components cluster aware. In my next post I will be peeking into the strategies with respect to the individual JBI components other than the standard ones like HTTP/ORB/JMS.
When we speak of AppServer the following 3 components come to our mind.
Web Server – catering to the http requests
EJB container – catering to the ORB requests and
MQ – catering to JMS
When we observe Glassfish each of the component has its own strategy for clustering and load balancing. Each of the strategy vows its implementation either to its legacy (meaning its already there like in case of http load balancer) or need arising out for that specific protocol in a given scenario. We can observer variance with respect to the same protocol being implemented with different strategies, since not being owners as in case of JMS. This is because of the nature of the protocol. For example, if we take inbound scenario, HTTP gets connections from various sources and there can be symultaneous requests at the same time so it needs in the facade in the load balancer, but in the case of JMS / JDBC etc. the inbound is from one source. Its push from the browser in the former case where as its a pull in the other case (The MDB mechanism or the inbound message processor polls the data).
I tried to explain each of the strategy for the above protocols with respect to Glassfish, and concluded that each protocol must come up with its own strategy and the same can not be generalized for all the protocols because of the sheer variety of them and the need of each protocol. Though we can generalize certain aspects and can bring forth the LCM of all the protocols at a protocol agnostic way the time and effort might not match the same.
I also tried to discuss different strategies that we can adopt for making the components cluster aware and having the load balancing and failover facilities in a generic way. I also tried to explore the API so that the BC while writing the cluster aware code can get the details and info required of the cluster.
Lets Look at the Clustering environment of the AppServer first.
Resources: Each Resource accessed by applications reference external resources such as JDBC database resources and their associated connection pools, CMP persistence managers, JMS resources, java mail resources, custom JNDI resources, and connector resources. Like an application, each resource has a JNDI name which is unique across the domain. An un-clustered server instance or cluster can reference zero or more resources, and a resource can be staged in which case it is referenced by no server instances.
How does AppServer handle Clustering and Load Balancing?
This understanding is needed so that we can be aware of how things happen at the
AppServer side and how can they be extrapolated in the JBI environment.
For AppServer there are 2 perspectives with respect to clustering:
Administrative -- from this stand point cluster is a bunch of
homogeneous machines/Server Instances for the DAS
Per Instance – this is for individual instance need.
- Load balancer
- Route the request. – this mechanism works like having context root – port mapping which helps in routing the request.
- Maintaining sticky session -- the LB acts as façade for HTTP requests that are coming in and maintains a session store in HADB for performance reasons and for optimization.
Lets look at IIOP/ORB:
1. Sun’s own MQ: In this case, when the MDB end point gets activated, it is set with the cluster aware setting (ClusterName+InstanceName+MDBName). This actually helps as a hook to the instance on which the MDB is listening for the JMS. This helps the MDB maintains the stickiness by way of maintaining Local Delivery Profile (LDP).
How are the transactions handled?
Every server instance is having its own Transaction Manager. This TM actually maintains the list of transactions that are handled. It also maintains the log that is pertaining to the transaction. If an application instance is killed during the transaction the TM pertaining to that server instance will take care of that transaction. Since transactions are atomic in nature the resource recovery (commit/rollback) would not be effected. There would not be a case where in a transaction is started by one instance and had to be dealt by other instance.
What modules are available for supporting Clustering in AppServer?
1. Group Management Service (GMS): GMS is an independent software module, from Project Shoal (https://shoal.dev.java.net), which may be embedded and started by processes that require runtime cluster communications and group management services such as:
1. Static Group Membership Composition change notifications :
a. Member Added Notification
b. Member Removed Notification
2. Dynamic Group Membership Composition change notifications :
a. Join Notifications
b. Failure Suspicion Notifications
c. Failure Notifications
d. Planned Shutdown Notifications
3. Recovery Oriented Support Services:
a. Delegate Recovery Instance Selection and notification
b. Protecting recovery operations through failure fencing
4. Messaging Service API for Group and Member-to-Member messaging
5. Distributed Caching of lightweight state and recovery states
- Failure Notifications
- Recovery member selection and corresponding notifications
- Failure Fencing
- Member Joins and Planned Shutdown Notifications
- Support for administrative configurations
- Group, One-to-Many and One-To-One Messaging
- A Distributed State Cache implementation to store data in a shared cache that lives in each instance's GMS module.
GMS provides a simple, easy-to-use API to its clients for accessing and consuming its functionalities. GMS provides a Group Communication Service Provider Interface for group communications provider technologies to be integrated. In our implementation, we use a Service Provider implementation based on JXTA peer-to-peer technology to construct the desired group communications infrastructure.
2. Load balancer Module:
AppServer study Conclusion: