Thursday Jan 25, 2007

GlassFish v2 Milestone 4 includes Shoal

GlassFish v2 has reached its Milestone 4 which is the Hard Code Freeze for the Beta. During this time, we addressed many bugs and features in Shoal and delivered a stable version to GlassFish

Shoal gets autoenabled when a cluster is created (Of course, one can disable it by setting the cluster's heartbeat-enabled attribute to false). When a cluster is created, the Domain Admin Server creates an instance of this cluster as a group and joins it as a Spectator member. So for each cluster, the Domain Admin Server participates as a Spectator member.

After instances are added to cluster and as these are started, each instance in the cluster participates in this Shoal cluster and can then communicate with each other and monitor each other's state. When an instance or cluster is shutdown administratively, Shoal notifies all instances about this impending shutdown allowing Shoal GMS client components within each instance to do appropriate pre-shutdown operations. Similarly, when an instance fails, Shoal notifies client components in all surviving instances about this failure and if needed to notify the need to perform any recovery operations. Details of these features are available at the Shoal site.  

Shoal is used in GlassFish for group membership monitoring and for value add features like automated transaction recovery performed by a delegate instance, self management wherein administrator can define remedial actions based on occurence of events including group events that Shoal notifies. The IIOP Load Balancer uses Shoal for dynamically generating failover IIOP end points as cluster shape changes and provides this to IIOP clients out-of-band so that these have updated failover targets. The In-Memory Replication component uses Shoal to confirm failures and to determine when a member has joined or gracefully left the group.

By GlassFish FCS, we will make Shoal stable enough to be released on its own as a standalone infrastructural component that other projects/products can use.

Shoal presents some interesting product use cases.

An in-memory cache on the lines of memcached should be a little more work and is a good possibility. A Shoal-based compute grid is another possibility with relatively small incremental code to be written to make that happen. I can also think of a Derby cluster using Shoal as the underlying clustering and replication mechanism. Once you think of Derby, any other product could be applied here. For instance, a directory server cluster, a web server cluster, a message queue cluster, etc. Another possibility is that Web-only apps could directly use Shoal to cache states among instances of web containers. So you see, the possibilities are endless and is only limited by imagination. :)

Thursday Nov 09, 2006

Introducing Project Shoal : another open source contribution from Sun

Project Shoal was started in the java-enterprise community incubator a few months ago. We are happy to report that over the last week, we have commited sources to the project's CVS repository. The sources are reasonably well tested and are a good starting point towards building a quality product over the ensuing months.

The project's goal is to produce a Java-based dynamic clustering framework that can be plugged into any product requiring clustering functionality as an in-process component.  One can think of several important use cases that such a library will serve ranging from basic group membership service to building fault tolerance and reliability oriented solutions to distributed caches to high availability  infrastructure.

The heart of Shoal lies in its Group Management Service which provides a group membership management infrastructure such as group and member discovery, detecting failures, planned shutdowns, etc. in addition to value add features such as recovery oriented support and lightweight caching. Group members can also send messages to an individual member, a collection of members or all members. Group members can be cluster or non-cluster members identified by their member type. Members communicate by simply using their application level member identity and do not have to know anything about the network level locational details of themselves or other group members.

Shoal exposes an easy-to-use client API for consuming client components in each JVM process and provides an effective abstraction shielding clients from complexity of bootstrapping into a process group and its networking semantics. This, it achieves, through a service provider interface (a still-evolving SPI) that allows group communication technologies to be integrated. The default communication provider is based on JXTA peer-to-peer technology.

During the last few months, while the internal approval processes were ongoing, we built a very productive relationship with the JXTA community. JXTA has several inherent strengths in the group communication space by virtue of its being in the peer-to-peer (p2p) area.  JXTA provides very good security (authentication and encryption) support allowing pluggable keystores, dynamic route mapping (especially useful when peers are mobile), transport agnosticity (uses virtual multicast through rendezvous services if UDP multicast is not supported, TCP or HTTP transports that are dynamically switchable when a given transport is unavailable), WAN capable, simple addressing semantic (application instance name encoded peer identifiers), etc.


Shoal is part of the GlassFish Community and will be incorporated for value added features in GlassFish v2. GlassFish code base forms the core matter within the Java EE SDK and the Sun branded and supported Sun Java System Application Server 9.x.


Watch out for more blogs on Shoal in the coming weeks. We welcome interested folks to join the project and contribute to the success of Shoal in various ways including contributing code, bugs, patches, documentation, spreading the word, code reviews, etc.

Here are some Shoal related blogs:

by Mohamed Abdelaziz  (JXTA)

by Bernard Traversat (JXTA)

Blog by
Masood (Max) Mortazavi (Java DB)



Shreedhar Ganapathy


« October 2016