Sailfin drives a new feature in Shoal: JoinedAndReadyNotificationSignal

As usage of Shoal's clustering framework increases across products, new feature requests are coming in to enrich Shoal's offerings for employing applications.

The Sailfin project is building a Telecommunications Application Server with contributions from Ericsson and Sun. One of the parts of Sailfin is a Java based load balancer called ConvergedLoadBalancer (CLB). The CLB load balances both Http and SIP based requests. The CLB is unique in that it is a component of the Sailfin appserver instance. Thus, any instance in the cluster can be configured as a load balancer. Such an instance can be part of a Front End LB Cluster performing load balancing of requests on a separate application server cluster tier or it can be part of a Self LoadBalancing Cluster wherein one or more instances of the application server cluster also perform the role of a load balancer while also serving requests. The CLB component and other Sailfin components employ Shoal for cluster events, messaging and health monitoring.

The CLB (as with any load balancer) needs to know when instances of a cluster have not only joined a cluster but also when the instances are ready serve requests. Shoal was up until now providing a JoinNotificationSignal which would be triggered as soon as each instance in the cluster used Shoal's GMS to join the cluster. This was sufficient for many use cases but for the LB it needed to know not just that but also when the instance had completed startup operations. This requirement helped us design a new notification called JoinedAndReadyNotificationSignal that would be disseminated to the cluster members for each instance completing startup and reporting such a completion to the group.

Shoal's GroupManagementService provides a new API called reportJoinAndReadyState() which the employing parent application can now call when the parent application has completed its own initialization and startup indicating a point where operational activities on the instance can now start.

Sheetal and I recently committed the code supporting this new feature.

This feature can be particularly useful while building Compute Grid and Cloud Computing type services using Shoal wherein each Grid node can now report when they are ready to act as nodes in the grid. BTW Shoal's concept of  a GroupLeader ties in well with the Grid's Compute Task Manager abstraction and is a good infrastructural fit.

Do please share your feedback with us at users alias about what you would like to see added in Shoal to serve your clustering and fault tolerance needs. Shoal's goal of clustering goes much beyond the realm of Data Grids (which we are looking into building ) with a wider  spread in terms of building fault tolerance solutions.


Hi, my name is John Kim.

I have some problems with Shoal and I tried here and there for that, but failed.

Is there any configuration to set Shoal to use one of the multiple network interfaces on the server?

Thanks in advance.

Posted by John Kim on January 31, 2008 at 09:22 AM PST #

Hi John,
Thanks for posting the comment. What problem are you facing? Could you describe it in some detail?


Posted by Shreedhar on January 31, 2008 at 10:13 AM PST #

You can also post your question to our users mailing alias users[at]shoal[dot]dev[dot]java[dot]net
That way more people can see your post and respond.

Posted by Shreedhar on January 31, 2008 at 12:03 PM PST #

Hi Shreehar,
Thanks for the quick response. My problem is that I have two network interfaces on the server. One network interface is for private network and the other is for public network. I want to set Shoal to use one of the network interfaces, which is, in this case, the latter one - the network interface for the public network. When I just start Shoal, it simply does not work with two network interfaces. I think there should be some thing like java -Djgroups.bind_addr=, this is one example from JGroups.


Posted by John Kim on January 31, 2008 at 01:22 PM PST #

Thanks John for your feedback.

This is not currently implemented in Shoal. We have opened an RFE to fix this :

Please feel free to post any questions/comments/suggestions to improve Shoal at our users mailing alias : users[at]shoal[dot]dev[dot]java[dot]net

You can also file defects/enhancements in the Issue Tracker at


Posted by Sheetal on February 01, 2008 at 03:42 AM PST #

the provider supports it, however a property must created and set in order to enable such feature.

see issue :

Posted by Mohamed Abdelaziz on February 01, 2008 at 03:46 AM PST #

Hi Shreehar,
I already tried using Jxta's setTcpNetworkInterace and setHttpNetworkInterface by modifying the Shoal source. It worked just fine. However working on Shoal source level is something I try to stay away from for compatibility issues later.
Another question is that Shoal uses 9999 as TcpEnd port. That means, if I use that port as a listening port, there could be some problem like port conflict or something?

I am sorry for not using the mail alias this time. I will do it next time.

I have been using JGroups for quite some time for our products. I found that Shoal would be better fit for us. Right now, I am trying to find out more about Shoal in terms of the architecture and the scalability...



Posted by John Kim on February 01, 2008 at 09:33 AM PST #

Hi John,

I just checked in the fix for this issue (issue 40). Please try it out and let us know.

You will need to set the following property in your code :

Properties props = new Properties();
props.put(ServiceProviderConfigurationKeys.BIND_INTERFACE_ADDRESS, "network_interface_to_bind_to");
GroupManagementService gms = (GroupManagementService) GMSFactory.startGMSModule(serverName, groupName, memberType, props);

Please checkout the latest Shoal workspace and build it (simply execute "ant" under <Shoal_workspace>/shoal/gms/).
The jars will be available at :
Please put these jars in your path.

We would love to hear from you !


Posted by Sheetal on February 08, 2008 at 07:57 AM PST #

Hi Shreedhar,

This is more an architectural query regarding the usage of Shoal in an adaptive load-balancer.

Currently, my scheme allows individual instances to which requests have to be forwarded using a weighted-preference mechanism. ie. during startup, symmetric servicing components register their "capacity" with the load-balancer+message-router component. Load-balancer-router forwards incoming requests to individual service components based on registered weights.

Now, I am looking for an adaptive option of changing the above as follows:
a) Service components and load-balancer joins a group using GMS
b) My first query here - does the join notification allow carrying of some application specific custom meta-data packed with Join-Signal so that some initial capacity can be registered by service components to load-balancer during join-event (without requiring an additional group message to GMS group)?
c) Adaptively depending upon the load-conditions a particular service component currently handles it emits a "failure notify" to the GMS group - though the component technically does not fail, in this context it tries to prevent itself from being overloaded beyond its configured capacity by advertising to the group it has "failed"
d) When the the component which emitted "failure notify" - as above, overcomes its overload condition, it advertises/emits back "Join" / "Joined And Ready" back to GMS group.

Since Load-balancer effectively receives the notifications described in (c) and (d), it can update its "available" message recipients accordingly (removing from its list while receiving "failure notify" signal and adding to list when it receives "join" signal) and perform message-routing accordingly.

Overall, due you see any pitfalls in using Shoal for such an approach - atleast I do not see any :) ... May be you can highlight some cautions which would help me tread a bit carefully ..


Posted by Muthukumaran on October 17, 2008 at 09:57 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed

Shreedhar Ganapathy


« April 2014