Wednesday Mar 12, 2008

GlassFish High Availability Session at Sun Tech Days Hyderabad

At the recent Sun Tech Days event at Hyderabad, I gave a talk covering GlassFish's High Availability features, particularly the In-Memory Replication support, as part of GlassFish Day (Feb 29th). 

I had the privilege of talking to a full house of around 500 people. The session covered introduction to HA, how easy it is to create, and configure a cluster of instances, and to configure the application for enabling in-memory replication based availability. The session elicited very good questions ranging from the basics to involved ones in the area of sizing the heap to sticky sessions support. I spent an hour after the session outside the hall answering questions posed by interested folks from several companies.

Many attendees wanted to get a copy of the slide deck. Look here for it.

Needless to say, we would very much appreciate any feedback or questions on GlassFish's High Availability. Please send these to us at the GlassFish user mailing alias.


Tuesday Mar 11, 2008

Notes from GlassFish Booth at Sun Tech Days, Hyderabad '08

I am back after a two week visit to India. I took the week off last week to be with my folks and to unwind and recharge in my home city, Mumbai.

The week before that was an incredible one for me as I saw the hi tech boom in India first hand. The swelling and enthusiastic crowds at the Sun Tech Days event at Hyderabad on Feb 27, 28 and 29 was very thrilling to experience to say the least. The event was highly successful in attracting budding youngsters and experienced professionals alike. I am told that the content of the event was very fulfilling for attendees.

Some notes of interest :

  • Each session had between 1000 and 1500 attendees. Community Day was on Feb 29th with tracks such as GlassFish, OpenSolaris, Netbeans, etc. and each track had an expected 450 people based on hall capacity. In the GlassFish track, we saw that increase to around 500.
  • At the GlassFish booth, I met quite a lot of visitors many of whom had either vaguely heard about GlassFish or never heard of it. About a 3rd of people were ones who work with an Application Server in a professional capacity. Others were developers learning the ropes at work or students who were excited to see an open source appserver like GlassFish.
  • GlassFish is relatively new here - people have heard about it and are now beginning to try it out. So this was a huge opportunity to spread the word about GlassFish as a compelling open source project delivering a fully Java EE 5 compliant application server packing a whole bunch of features that commercial vendors usually provide for the cost of an expensive license and services attachment.
  • Common theme of questions from professionals were on availability of migration tools to move from BEA, and Websphere. They are looking for detailed documentation and engineering services support for migration to enable them to recommend GF in their orgs and in their client orgs. We are working on this on multiple fronts and through the migrate2glassfish project.
  • Many professionals I talked to had tried JBoss before but due to their employer or customer platform preference, they were on WebSphere or WebLogic.
    They were impressed with GlassFish's Administration ease of use and GlassFish's published SpecJ numbers.
  • Companies from which people came to booth read like the who's who of the hitech majors' list such as Wipro, Satyam, Infosys, Cap Gemini, etc.
  • Most users were still developing on J2EE 1.4. Many are beginning to move to Java EE 5.
  • Many developers I met were on Eclipse. I demoed GlassFish and NetBeans 6.0 when possible at the booth and many went back saying they will surely try NB with GF.
Based on reactions and questions that came from people at the booth, here are the things that I thought attracted many of these folks to think of converting from their existing middleware to GlassFish : 

 

  • GlassFish's open source status
  • Free for development and deployment
  • No license fee for product purchase
  • Strong community support
  • Sun's backing and commercial support offering (the concept of indemnification was new for some but many understood impact for their customers abroad)
  • Administration ease of use
  • Market leading product differentiators such as Grizzly, Virtual server support in Web tier, Web Services support, Easy cluster creation and management, Call Flow Monitoring, High Availability Options through In-memory replication and HADB, Inclusion of a quality MQ product, Netbeans and Eclipse IDE integration, etc.

Many of the professionals working with other application servers said the Administration, Clustering, High Availability, Loadbalancer support, Webservices support, migration support, and IDE integration would motivate them to try out or switch to GlassFish. 

Looking forward to hearing from these new GlassFish users at our user community mailing list.

 

 

 

Thursday Jan 17, 2008

Sailfin drives a new feature in Shoal: JoinedAndReadyNotificationSignal

As usage of Shoal's clustering framework increases across products, new feature requests are coming in to enrich Shoal's offerings for employing applications.

The Sailfin project is building a Telecommunications Application Server with contributions from Ericsson and Sun. One of the parts of Sailfin is a Java based load balancer called ConvergedLoadBalancer (CLB). The CLB load balances both Http and SIP based requests. The CLB is unique in that it is a component of the Sailfin appserver instance. Thus, any instance in the cluster can be configured as a load balancer. Such an instance can be part of a Front End LB Cluster performing load balancing of requests on a separate application server cluster tier or it can be part of a Self LoadBalancing Cluster wherein one or more instances of the application server cluster also perform the role of a load balancer while also serving requests. The CLB component and other Sailfin components employ Shoal for cluster events, messaging and health monitoring.

The CLB (as with any load balancer) needs to know when instances of a cluster have not only joined a cluster but also when the instances are ready serve requests. Shoal was up until now providing a JoinNotificationSignal which would be triggered as soon as each instance in the cluster used Shoal's GMS to join the cluster. This was sufficient for many use cases but for the LB it needed to know not just that but also when the instance had completed startup operations. This requirement helped us design a new notification called JoinedAndReadyNotificationSignal that would be disseminated to the cluster members for each instance completing startup and reporting such a completion to the group.

Shoal's GroupManagementService provides a new API called reportJoinAndReadyState() which the employing parent application can now call when the parent application has completed its own initialization and startup indicating a point where operational activities on the instance can now start.

Sheetal and I recently committed the code supporting this new feature.

This feature can be particularly useful while building Compute Grid and Cloud Computing type services using Shoal wherein each Grid node can now report when they are ready to act as nodes in the grid. BTW Shoal's concept of  a GroupLeader ties in well with the Grid's Compute Task Manager abstraction and is a good infrastructural fit.

Do please share your feedback with us at users alias about what you would like to see added in Shoal to serve your clustering and fault tolerance needs. Shoal's goal of clustering goes much beyond the realm of Data Grids (which we are looking into building ) with a wider  spread in terms of building fault tolerance solutions.


Friday Jan 11, 2008

GlassFish Hidden Nugget: Automatic Distributed Transaction Recovery Service

GlassFish v2 and v2 ur1 releases (and later) have support for transaction recovery (both manual and automated) in the sense that incomplete transactions at the time of an instance failure can be committed either manually or automatically.

Part of the new feature set in the cluster profile is a little known feature called Automated Distributed Transaction Recovery that comes out of Project Shoal's support for it. 

Essentially, Automatic Distributed Transaction Recovery in GlassFish works as follows :

Consider the following :

  • a cluster of three instances : instance1, instance2, and instance3
  • Two XA resources used by each GlassFish instance
  • a transaction starts on instance 1,
  • Transaction Manager on instance1 asks resource X to pre-commit,
  • Transaction Manager on instance1 asks resource Y to pre-commit,
  • Transaction Manager on instance1 asks resource X to do a commit,

Now, instance1 crashes

The Transaction Service component in one of the surviving members, instance2 and instance3, gets a notification signal that a failure recovery operation needs to be performed for a instance1. This signal from Shoal is called FailureRecoverySignal.

This notification signal comes to the Transaction Service component in only one particular selected instance as a result of a selection algorithm run in Shoal's GMS component that takes advantage of the identically ordered cluster view provided to it by the underlying group communication provider (default provider is Jxta).

The Transaction Service component in this instance, say instance2, would now go into its autorecovery block. It starts by waiting for a designated time (default to 60 seconds) to allow for the failed instance1 to start back up.

If instance1 is starting up, its own Transaction Service component would do self recovery to complete phase 1 transactions.

In instance2, after the wait timeout occurs, the transaction service component would now see if instance1 is part of the group view and if not try to acquire a lock for the failed instance's transaction logs through Shoal's FailureRecoverySignal and if successful (indicating that the failed instance did not startup), acquire the transaction log and start recovery of transactions i.e complete the commit operations for the pre-commit transactions. If the acquisition of the lock fails, then it gives up, and checks that the failed instance did startup through Shoal's group view and logs this fact.


If, during the recovery operations  being performed by instance2, the failed instance1 starts up, the transaction service component in this instance would first check with Shoal if a recovery operation is in progress for its resources by any other instance in the group and if yes, it waits for the recovery operations to be completed and then completes startup. This ability to check for such recovery operations in progress is through a related Shoal feature called Failure Fencing[1].  If there are no recovery operations in progress, then the startup proceeds with a self recovery which recovers any incomplete transactions in instance1's logs.

Now during recovery of instance1's transaction logs, instance2 fails, then the fact that this instance was in the process of recovering for instance1 is known to the remaining members of the group (i.e. instance3) through the failure fencing recovery state recorded in Shoal's Distributed State Cache. As a result, when instance3's transaction service gets the failure recovery signal, not only does it get it for instance2's failure, but also for instance1. This facility covers for cases where cascading failures or multiple failures occur.

Note that, for the automatic distrbuted transaction recovery to work, access to the transaction logs for all instances in the cluster for
purposes of auto recovery requires that the logs be mounted on a shared/mirrored disk[2].


[1] More on Shoal's Automated Delegated Recovery Selection
[2] Distributed Transaction Recovery

 

 

 

Saturday Dec 08, 2007

Excellent Article on Shoal by non-Sun authors

Just came across this excellent introductory article on Shoal clustering framework on Java.net which I believe is to be published on upcoming Tuesday going by the date posted (12/11/2007).

Noticeably, this is an article by authors that we, at the Shoal community, have not yet corresponded with. This is great news as it lets us know that there is a quiet adoption of this framework. The article lucidly explains salient aspects of Shoal's clustering approach and how easy it is to integrate it into your application/infrastructure. 

We hope this will make it even easier for users to adopt this technology.

Do send us your questions at the Shoal users mailing list.


Wednesday Dec 05, 2007

Shoal Whitepaper on Scalable Dynamic Clustering

Its been a while since I have blogged. I have been rather busy managing the GlassFish v2 ur1 release which is round the corner around Dec 13/14.

Meanwhile, Mohamed Abdelaziz and I put together a Whitepaper that gives details on the scalability and dynamic clustering aspects of Shoal. The paper provides a good overview of the self composing nature of Shoal and dives deeper into the set of protocols that provide the basis for building fault tolerant infrastructures.

Shoal is coming along very well in various spaces going by the increasing hits to our website from Telco and Financial majors. We are continuously improving Shoal into a robust component based on feedback from our user community. So keep'em coming to the users mailing list.

Tuesday Aug 14, 2007

Shoal Clustering User Guide Part 1

In this series of blog entries, I will provide a guide for new users on how to use Shoal for your application.

This particular blog entry provides a Shoal Clustering 101 type basic introduction. Subsequent blogs will enable the user to get more closer to the metal on how to use this technology.

What is Shoal? 

Shoal is a technology/framework that allows consuming applications the ability to participate in a group and thereby communicate with the group, register interest in being notified of group events, and share application data across group members. These functionalities enable Shoal to be used as a clustering framework in enterprise infrastructure software while also being capable of other use cases.

Shoal's core is the Group Management Service (GMS) which provides client APIs to interact with a group while allowing group communication libraries to be integrated through a service provider interface implementation. Currently with a Jxta service provider, Shoal takes advantage of advanced Jxta features for robustness, reliability and scalability.

While Shoal itself is scalable to many instances in a group (upto 64 nodes in our tests), the scaling is always determined by the size and characteristics of the employing application/product.

Shoal Downloads  

Among the first things you want to know is where to get Shoal downloads.  

Shoal download is available here. Pick the latest zip file for the latest and greatest stable version. The zip file contains two jars of interest - the shoal-gms.jar containing Shoal's client api, GMS core implementation, and a Jxta service provider implementation,  and jxta.jar which is the Jxta peer-to-peer platform.

Shoal Documentation 

Now that you have the jars, you would need documentation to see how to integrate Shoal into your product. Shoal offers APIs to consuming applications to participate in a cluster. Shoal's JavaDocs containing the APIs are available here. In the JavaDoc, select the com.sun.enterprise.ee.cms.core package.  Look in the Description section for a simple introduction to the API.

As the Description mentions, the GMSFactory class located in the com.sun.enterprise.ee.cms.core package is the entry point for getting Shoal's Group Management Service.

Code Snippet 

The following code snippet uses the GMSFactory to start the GMS module, use the GroupManagementService reference to initialize the group, register interest in events,  join the group and the api to call when the process is ready to leave the group : 


//initializes GMS and underlying group communication provider

final GroupManagementService gms = GMSFactory.startGMSModule(serverIdentifierName, groupIdentifierName, GroupManagementService.MemberType.CORE, configProperties);

//register for group events

//register to receive notification when a process joins the group
gms.addActionFactory(new JoinNotificationActionFactoryImpl(this));

//register to receive notification when a group member leaves on a planned shutdown
gms.addActionFactory(new PlannedShutdownActionFactoryImpl(this));

//register to receive notification when a group member is suspected to have failed
gms.addActionFactory(new FailureSuspectedActionFactoryImpl(this));

//register to receive notification when a group member is confirmed failed
gms.addActionFactory(new FailureNotificationActionFactoryImpl(this));

//register to receive notification when this process is selected to perform recovery operations on a failed member's resources
gms.addActionFactory(serviceName, new FailureRecoveryActionFactoryImpl(this));

//register to receive messages from other group members to this registered component

gms.addActionFactory(new MessageActionFactoryImpl(this), componentName);

//joins the group
gms.join();
 

//leaves the group gracefully
gms.shutdown(GMSConstants.shutdownType.INSTANCE_SHUTDOWN); 

As the above code snippet shows, it is extremely easy to instrument Shoal into your application and be on the road to clustering your application and take remedial actions on occurrence of group events.

A Simple Shoal Example (sources)

Look through this example code to see the full source of a Simple Shoal Example.  Look in the runSimpleSample() method to understand steps to make Shoal an in-process component.

Questions? Comments? 

If you have questions on the above, please send your questions to the Shoal users alias: users [at] shoal [dot] dev [dot] java [dot] net or post your questions as a comment to this blog entry.

In the next blog entry, I will go a bit deeper explaining specific parts of Shoal.


Tuesday Aug 07, 2007

Shoal Clustering Framework 1.0 Early Access available

Its been a fairly long time since I blogged.

Over the past few months, we (the GlassFish HA team and the Jxta Team ) have been concentrating on improving and addressing GlassFish high availability features and associated bugs. In the process, Shoal's Group Management Service benefited from intensive QE cycles on 8-node GlassFish clusters under scores of scenarios and test cases. We have been focused on fixing bugs, improving performance, and progressively gating changes in Shoal to manage risks for the upcoming GlassFish v2 FCS release. 

At the moment, Shoal is in good shape and we decided to release the Shoal 1.0 Early Access version a couple of days ago. This will be followed by the 1.0 Final release before the GlassFish v2 release once we know that we have delivered the final acceptable bits for that product.

After that our next step is to address the unique requirements that arise out of the Sailfin project which is building a SIP supported application server based on GlassFish.  

We'd love to hear feedback from our user community about your success stories, issues and enhancement requests using Shoal.

We have seen anecdotal evidence of how useful and easy-to-use this library is and it would help improve our adoption and project growth if more specific feedback is available. Our Statcounter statistics are showing companies from very interesting industry segments going through the Shoal site, downloads and documents so this is a huge boost to our commitment to build a good group communications API based library.

If you have a success story or feedback using Shoal that you can share, drop us a line at the Shoal user alias

users[at]shoal[dot]dev[dot]java[dot]net

and we will highlight it our blogs and on the Shoal web site.

Also we welcome code and design contributions from experienced clustering and distributed systems developers.

 

 

 

Tuesday Jun 12, 2007

Safari on Windows XP, Initial thoughts

Just downloaded Safari beta for Windows XP and was earlier writing this blog entry using it before moving back to FireFox.

The Javascript widgets at the blogs.sun.com editor do not show up as this is not a supported browser. I wish Roller would show the widgets on a best effort basis.

Upsides:

The rendering quality and speed is awesome. Absolutely blazing fast on my home DSL connection. And the look and feel is very smooth and pleasing. Has Tabbed viewing support. Renders RSS feeds on a site with a single click and looks very nice. Has popup blocker, and most features that FireFox.

Downsides:

Commonly used sites sometimes dont work with Safari beta on Windows. Seems to have problems with forms. Hotmail, YahooMail, did not work in my experience (to its credit, web 2.0 sites such as dzone.com, upcoming.org worked well).

On this blog entry being written on Safari, in the end it did not allow me to post the blog or save it as draft. The buttons just would not work.

I hope these bugs get addressed soon.
Looking forward to the final version. Wish Firefox were as fast and smooth! Oh well, a little competition is good for all products.

 

Saturday Mar 31, 2007

NetBeans and GlassFish clusters:What are your needs?

We are looking for feedback from users of NetBeans and GlassFish on what they would like to see in terms of their runtime interactions through the IDE with cluster(s) in GlassFish.

Most IDEs provide the ability to work with a single instance of an application server such as GlassFish, etc.  I do not see the notion of cluster(s) in the IDEs unless I have missed something obvious. There must be various use cases wherein a developer (typically an advanced developer/deployer/tester such as a System Integrator) would need the ability to interact with a cluster through the IDE.

Clustering typically involved an environment consisting of many instances that can be located on different remote machines. Such instances are created/started/stopped/deleted on each machine by a dedicated agent process that follows instructions from an administrative server within which a cluster is a logical configuration entity while instances are physical entities. So, within the IDE, there are possibly but a few operations that can be executed/managed.

So here are some questions for which I'd appreciate responses through comments (Add more if you can think of any further):

  1. Is there indeed a need for interacting with clusters through the IDE?
  2. What problems do you see today with respect to clusters from within your IDE?
  3. If the browser based admin console of the appserver can be lauched from within the IDE would this  be sufficient to address your needs?
  4. If not, why is the IDE an important piece to help solve that problem?
  5. If interacting with cluster(s) is important, what should that interaction functionality be from within an IDE?
  6. If debugging seems like an obvious use case, would that not already be addressed by working with individual instances that are part of a cluster?
  7. What other use cases do you work with that you wish an IDE could provide when working with clusters?

Do add more questions and answers if you can think of more.

Tuesday Mar 27, 2007

Article on Introducing PetStore 2.0

Blueprints team members Inderjeet Singh, Mark Basler and Sean Brydon recently teamed up with Sun staff writer Dana Nourie  for an article Introducing the Java PetStore 2.0 Application.

There is a lot of interest among Java developers to learn how next generation dynamic web technologies such as Ajax can be put to use in Java EE applications that deploy in an enterprise setting. The article has been very popular and generated many thousands of hits since publication a couple of weeks ago. The PetStore Blueprint application's value proposition to developers is to show how one can use Web 2.0 technologies with end-to-end Java EE apps. 

The Blueprints project provides many such learning artifacts. A great way to learn more in this space is the Blueprints Solutions Catalog which is now hosted live with a view app at our web application developers' page. The viewer app is itself a Web 2.0 app and provides content organized into useful categories.  See more Web 2.0 samples here.

The Blueprints apps have been tested with various releases and builds of GlassFish. The Java EE SDK includes Blueprints apps so that is the easiest way to get both the GlassFish application server and the above applications in one download.


 

Wednesday Feb 14, 2007

Blueprints team delivers Web 2.0-based PetStore 2.0 application

The Blueprints team has delivered a brand new (early access but mostly complete)  version of the popular PetStore application.

What's different about this release is that this application demonstrates the use of latest emerging technologies (popularly called Web 2.0) and uses the more mainstream pattern preference of developers, that of a web-only application that employs the JPA for persistence. As a result, the PetStore 2.0 is a significantly lightweight yet rich web application that shows how to mesh Java EE 5 artifacts with emerging technologies such as AJAX, mashups, etc.


Petstore 2.0 can be viewed live on Sun's Developer Web site located at http://webdev1.sun.com/petstore/faces/index.jsp. The announcement of the release is located at https://blueprints.dev.java.net/servlets/NewsItemView?newsItemID=4632.


The Petstore 2.0 application demonstrates numerous Web 2.0 uses cases in the application including Ajax, Client-side mashups, Server-Side mashups, Delegated mashups, RSS news bar through ROME, File upload and much more. Petstore 2.0 is part of the Java EE 5 SDK and runs on Glassfish. The release notes for Petstore are located at https://blueprints.dev.java.net/petstore/release_notes.html.

If you are a developer looking for resources and guidelines on how to build AJAX and other web 2.0 technologies enabled Java EE apps, download the PetStore and review the code base. The code is released under the BSD license and is reusable in your apps.

You can also discuss your questions at the blueprints forum located at : http://forum.java.sun.com/forum.jspa?forumID=121 

 

Friday Jan 26, 2007

Differences between GlassFish v1 and GlassFish v2

I frequently see this search term "Differences between GlassFish v1 and GlassFish v2" appear in Google searches showing up on my Statcounter account referrer link. So I guess its a good topic to blog about.

GlassFish v1 was the first release of the Java EE 5 application server from the GlassFish Project at Java.net. This release had featured a application server environment where one could create a domain with one appserver instance in it that was Java EE 5 compatible and included a good looking administration console in addition to performance enablers such as the nio-based Grizzly http connector framework. This was released around JavaOne in 2006. At the time, the community asked for more given the positive responses we received on v1. The community asked about our roadmap and insisted that clustering capabilities were a dire need among other things. We listened to the community and came up with the plan for GlassFish v2. In the meantime, we have also released update releases to GlassFish v1 i.e ur1 and ur1 p01

GlassFish v2 development is in progress and has reached Milestone 4 ( the one we use for the beta release). Primarily, the v2 release offers ability to create a domain with many standalone instances and clusters of instances. In essence you could have one or more domains, each with its own appserver instances some or all of which may be part of one or more clusters (well, an instance can only be part of one cluster, but you can have multiple clusters each with its own instances). In addition, there is the HA component which provides in-memory session state persistence between instances in a cluster. Numerous improvements to the base v1 code have been made to make it more performant. We have dynamic clustering coming in from Shoal that enables components in a clustered instance, and the administrator to get clustered instance status. V2 also contains dynamic IIOP endpoints failover loadbalancer.

The other notable difference from v1 is that you can run GlassFish v2 in various profiles, namely, Developer Profile, Cluster Profile, or Enterprise profile. A domain can be created with a specific profile name. With Developer profile you get the single instance domain that is well suited for development mode. The difference between cluster profile and enterprise profile is sort of subtle. The enterprise profile carries almost all of what cluster profile offers except that it is aimed at large scale deployments in enterprises. The cluster profile basically allows you to create within your domain, not only non-clustered standalone instances but also clusters with instances attached to it with homogeneous configuration. In other words, one can have multiple clusters and have sets of instances within each cluster with homogeneous configuration. The instances can be spread over many machines and the lifecycle of these instances is managed by an entity called a Node Agent which sits on each machine and responds to the domain administrative server's instructions to stop or start an instance on a particular machine. One can deploy in a single action, applications to a target cluster and all instances in that cluster will have the application deployed along with corresponding resources.

The differences between these profiles are covered well in this One Pager document section 4.1.2

There is more as per this blog in the Aquarium :
 • Update Center
 • WS Interoperability (WSIT) Functionality
 • JBI Support
 • In-memory replication.

 • Dynamic clustering from Shoal.

 • Optimized ORB
 • Startup Architecture improvements.
 • Improved Web Tier: Comet,
virtual hosting, in memory JSP compilation.
 • JSF used in the Admin GUI.

Well, you get the idea on how much progress has been achieved from v1 last year to v2 this year.

 

Send in comments if you would like to hear more details and I will go dig around to get you the necessary information.

 Also send in your successful deployment stories on GlassFish to the Stories blog

 

Thursday Jan 25, 2007

GlassFish v2 Milestone 4 includes Shoal

GlassFish v2 has reached its Milestone 4 which is the Hard Code Freeze for the Beta. During this time, we addressed many bugs and features in Shoal and delivered a stable version to GlassFish

Shoal gets autoenabled when a cluster is created (Of course, one can disable it by setting the cluster's heartbeat-enabled attribute to false). When a cluster is created, the Domain Admin Server creates an instance of this cluster as a group and joins it as a Spectator member. So for each cluster, the Domain Admin Server participates as a Spectator member.

After instances are added to cluster and as these are started, each instance in the cluster participates in this Shoal cluster and can then communicate with each other and monitor each other's state. When an instance or cluster is shutdown administratively, Shoal notifies all instances about this impending shutdown allowing Shoal GMS client components within each instance to do appropriate pre-shutdown operations. Similarly, when an instance fails, Shoal notifies client components in all surviving instances about this failure and if needed to notify the need to perform any recovery operations. Details of these features are available at the Shoal site.  

Shoal is used in GlassFish for group membership monitoring and for value add features like automated transaction recovery performed by a delegate instance, self management wherein administrator can define remedial actions based on occurence of events including group events that Shoal notifies. The IIOP Load Balancer uses Shoal for dynamically generating failover IIOP end points as cluster shape changes and provides this to IIOP clients out-of-band so that these have updated failover targets. The In-Memory Replication component uses Shoal to confirm failures and to determine when a member has joined or gracefully left the group.

By GlassFish FCS, we will make Shoal stable enough to be released on its own as a standalone infrastructural component that other projects/products can use.

Shoal presents some interesting product use cases.

An in-memory cache on the lines of memcached should be a little more work and is a good possibility. A Shoal-based compute grid is another possibility with relatively small incremental code to be written to make that happen. I can also think of a Derby cluster using Shoal as the underlying clustering and replication mechanism. Once you think of Derby, any other product could be applied here. For instance, a directory server cluster, a web server cluster, a message queue cluster, etc. Another possibility is that Web-only apps could directly use Shoal to cache states among instances of web containers. So you see, the possibilities are endless and is only limited by imagination. :)
 

Thursday Nov 09, 2006

Introducing Project Shoal : another open source contribution from Sun

Project Shoal was started in the Java.net java-enterprise community incubator a few months ago. We are happy to report that over the last week, we have commited sources to the project's CVS repository. The sources are reasonably well tested and are a good starting point towards building a quality product over the ensuing months.


The project's goal is to produce a Java-based dynamic clustering framework that can be plugged into any product requiring clustering functionality as an in-process component.  One can think of several important use cases that such a library will serve ranging from basic group membership service to building fault tolerance and reliability oriented solutions to distributed caches to high availability  infrastructure.


The heart of Shoal lies in its Group Management Service which provides a group membership management infrastructure such as group and member discovery, detecting failures, planned shutdowns, etc. in addition to value add features such as recovery oriented support and lightweight caching. Group members can also send messages to an individual member, a collection of members or all members. Group members can be cluster or non-cluster members identified by their member type. Members communicate by simply using their application level member identity and do not have to know anything about the network level locational details of themselves or other group members.


Shoal exposes an easy-to-use client API for consuming client components in each JVM process and provides an effective abstraction shielding clients from complexity of bootstrapping into a process group and its networking semantics. This, it achieves, through a service provider interface (a still-evolving SPI) that allows group communication technologies to be integrated. The default communication provider is based on JXTA peer-to-peer technology.


During the last few months, while the internal approval processes were ongoing, we built a very productive relationship with the JXTA community. JXTA has several inherent strengths in the group communication space by virtue of its being in the peer-to-peer (p2p) area.  JXTA provides very good security (authentication and encryption) support allowing pluggable keystores, dynamic route mapping (especially useful when peers are mobile), transport agnosticity (uses virtual multicast through rendezvous services if UDP multicast is not supported, TCP or HTTP transports that are dynamically switchable when a given transport is unavailable), WAN capable, simple addressing semantic (application instance name encoded peer identifiers), etc.

 

Shoal is part of the GlassFish Community and will be incorporated for value added features in GlassFish v2. GlassFish code base forms the core matter within the Java EE SDK and the Sun branded and supported Sun Java System Application Server 9.x.

 

Watch out for more blogs on Shoal in the coming weeks. We welcome interested folks to join the project and contribute to the success of Shoal in various ways including contributing code, bugs, patches, documentation, spreading the word, code reviews, etc.

Here are some Shoal related blogs:


Blog
by Mohamed Abdelaziz  (JXTA)

Blog
by Bernard Traversat (JXTA)

Blog by
Masood (Max) Mortazavi (Java DB)
 

 


About

Shreedhar Ganapathy

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today