Shoal Clustering User Guide Part 1

In this series of blog entries, I will provide a guide for new users on how to use Shoal for your application.

This particular blog entry provides a Shoal Clustering 101 type basic introduction. Subsequent blogs will enable the user to get more closer to the metal on how to use this technology.

What is Shoal? 

Shoal is a technology/framework that allows consuming applications the ability to participate in a group and thereby communicate with the group, register interest in being notified of group events, and share application data across group members. These functionalities enable Shoal to be used as a clustering framework in enterprise infrastructure software while also being capable of other use cases.

Shoal's core is the Group Management Service (GMS) which provides client APIs to interact with a group while allowing group communication libraries to be integrated through a service provider interface implementation. Currently with a Jxta service provider, Shoal takes advantage of advanced Jxta features for robustness, reliability and scalability.

While Shoal itself is scalable to many instances in a group (upto 64 nodes in our tests), the scaling is always determined by the size and characteristics of the employing application/product.

Shoal Downloads  

Among the first things you want to know is where to get Shoal downloads.  

Shoal download is available here. Pick the latest zip file for the latest and greatest stable version. The zip file contains two jars of interest - the shoal-gms.jar containing Shoal's client api, GMS core implementation, and a Jxta service provider implementation,  and jxta.jar which is the Jxta peer-to-peer platform.

Shoal Documentation 

Now that you have the jars, you would need documentation to see how to integrate Shoal into your product. Shoal offers APIs to consuming applications to participate in a cluster. Shoal's JavaDocs containing the APIs are available here. In the JavaDoc, select the com.sun.enterprise.ee.cms.core package.  Look in the Description section for a simple introduction to the API.

As the Description mentions, the GMSFactory class located in the com.sun.enterprise.ee.cms.core package is the entry point for getting Shoal's Group Management Service.

Code Snippet 

The following code snippet uses the GMSFactory to start the GMS module, use the GroupManagementService reference to initialize the group, register interest in events,  join the group and the api to call when the process is ready to leave the group : 


//initializes GMS and underlying group communication provider

final GroupManagementService gms = GMSFactory.startGMSModule(serverIdentifierName, groupIdentifierName, GroupManagementService.MemberType.CORE, configProperties);

//register for group events

//register to receive notification when a process joins the group
gms.addActionFactory(new JoinNotificationActionFactoryImpl(this));

//register to receive notification when a group member leaves on a planned shutdown
gms.addActionFactory(new PlannedShutdownActionFactoryImpl(this));

//register to receive notification when a group member is suspected to have failed
gms.addActionFactory(new FailureSuspectedActionFactoryImpl(this));

//register to receive notification when a group member is confirmed failed
gms.addActionFactory(new FailureNotificationActionFactoryImpl(this));

//register to receive notification when this process is selected to perform recovery operations on a failed member's resources
gms.addActionFactory(serviceName, new FailureRecoveryActionFactoryImpl(this));

//register to receive messages from other group members to this registered component

gms.addActionFactory(new MessageActionFactoryImpl(this), componentName);

//joins the group
gms.join();
 

//leaves the group gracefully
gms.shutdown(GMSConstants.shutdownType.INSTANCE_SHUTDOWN); 

As the above code snippet shows, it is extremely easy to instrument Shoal into your application and be on the road to clustering your application and take remedial actions on occurrence of group events.

A Simple Shoal Example (sources)

Look through this example code to see the full source of a Simple Shoal Example.  Look in the runSimpleSample() method to understand steps to make Shoal an in-process component.

Questions? Comments? 

If you have questions on the above, please send your questions to the Shoal users alias: users [at] shoal [dot] dev [dot] java [dot] net or post your questions as a comment to this blog entry.

In the next blog entry, I will go a bit deeper explaining specific parts of Shoal.


Comments:

hi there,

thank you.

Do you have a comparison chart with JGroups ?

BR,
~A

Posted by anjan bacchu on August 15, 2007 at 12:22 AM PDT #

Hi Anjan
At the moment we don't have a comparison between JGroups and Jxta based service provider. Perhaps this is another blog topic.

The GMS API, though can be used such that underlying service provider is JGroups. We lack the resources to have an implementation to that effect yet but are looking for some community participation in that respect.

Expect a blog on this.

Thanks
Shreedhar

Posted by Shreedhar on August 15, 2007 at 12:35 AM PDT #

Hi,
I'm trying to use Shoal..But I'm missing something in the concept maybe...
I've used the example you suggested but it didn't work fro me..
I'm running the example under jdk 1.6, by invoking 2 instances of the main class..and there is no messages transfer between the 2 instances!
is it ok or something is totally wrong with what i'm doing ?
can you please guide me to make this example run as expected?
How Shaol is related to "Sun GlassFish Enterprise Server", if any?

Tnanks in advance.

Posted by Farid on May 11, 2008 at 01:08 AM PDT #

Hi Farid
Can you post this question at the shoal users mailing list along with the output shown on your console ? The mailing list is referenced in the Question/Comments section of the above blog.

Also could you post the output of ipconfig \\All (windows) or ifconfig -a (\*nix) ?

Shoal is related to GlassFish by way of it being the runtime clustering engine for that appserver. Also it is part of GlassFish community.

hth
Shreedhar

Posted by Shreedhar on May 11, 2008 at 01:35 AM PDT #

Thanks Shreedhar.
I've already posted the same question at the Shoal users mailing list.
I'll send another mail with the results of what you requested to do..

Posted by Farid on May 11, 2008 at 05:49 PM PDT #

I've tried the example on other machine and it worked! The other machine has a single network card, while the mine has 2!
can you post some performance statistics ?
for example, how many messages per second a Sender can deliver to Target (on average) ?
I've done my own test on my machine (2 GB RAM, 3.4 GHZ CPU, 2 core, Windows XP SP2)
and the results is : 500 msgs/second !!
I think it's a very disappointing result for me and for what i intend to use it.
What is your opinion?

Thanks.

Posted by Farid on May 12, 2008 at 11:37 PM PDT #

Hi Farid
These questions are better posted at the user mailing list as many users will benefit from such questions/responses. It will also help us hear from others in order to prioritize.

FWIW, our focus has been to get the correctness of group membership events done right with different combinations of failure induction and time considerations. This is critical for the two major products that use Shoal at this point, i.e GlassFish application server and the Sailfin Telecommunications app server. Other major users are coming on board at this point but so far performant messaging has not been one of their highest priorities.

As a result, our focus on performance on the messaging side has not been that great.

This is an area where we are looking for community help/contributions as well given our limited resources.
You could start by filing an issue at the issue tracker on this. If you have a performance test that you can contribute to us as an attachment to the issue, that would be of great help.

Also any analysis on where bottlenecks exist will help fix issues.

Cheers
Shreedhar

Posted by Shreedhar on May 13, 2008 at 12:02 AM PDT #

Additionally, to help with measuring messaging performance, you should calculate vm heap size that you should use. Its important to know what your individual message size is and number of messages you are sending per second. This will help calculate the total heap and the rate of growth of heap. This heap growth rate can be gathered using JConsole. Setting heap size (-Xmx -Xmn) and stack size (-Xss) is very important for getting throughput.

Posted by Shreedhar on May 13, 2008 at 12:11 AM PDT #

Hi,
First of all, I want to thank you for quick response.
I've sent some previous question to the user mailing list, but i don't see them posted here..
is there any other place where i can find the Q/A sent to the user mailing list?

Thanks.

Posted by Farid on May 13, 2008 at 12:39 AM PDT #

Hi Farid
You should find your mails and responses in the archive page :
https://shoal.dev.java.net/servlets/BrowseList?listName=users&by=date&from=2008-05-01&to=2008-05-31&first=1&count=5

Better still just add yourself as a subscriber to this list using your java.net id.

Looking at the archive could you tell me if one of your messages has not be delivered?

Thanks
Shreedhar

Posted by guest on May 13, 2008 at 12:44 AM PDT #

Hello Shreedhar,
Can you provide some example for sharing data among cluster. Is there any book/document available to read about shoal in detail.

Thanks
Vishawjeet Saini
Khushii IT Solutions pvt. ltd.
Gurgaon,Haryana,India.

Posted by Vishawjeet on June 16, 2008 at 04:01 PM PDT #

SimpleShoalSample worked well within the same subnet, but under a second try, did not send any message to the another server when executed from two telecom operator, that is two ip networks.

Is the shoal cabable to working properly with 2 or more servers/vms from different ip addresses, like 154.2... and 132.1.... ? And if yes, where to find out those configurations?
And if no, which framework should I use in my case?

Thanks in advance,
Tuomas

Posted by tuomas on October 14, 2008 at 09:57 PM PDT #

Hi Tuomas
It is capable of what we call the cross subnet group communications. Recently however a bug introduced in this functionality has rendered it not in working condition, we have to attend to it.

Please see the following wiki page for instructions on how to configure this.

http://wikis.sun.com/display/shoal/Shoal+How-To+-+Configuring+for+cross+subnet+support

Also, you can send in your questions to the Shoal users mailing list : users [at] shoal [dot] dev [dot] java [dot] net

Posted by Shreedhar Ganapathy on October 15, 2008 at 12:12 AM PDT #

Hello,

Do I require any other framework for load balancing, if I am using Shoal as Clustering?

Or does Shoal has load balancing support as well?

Thanks,
Ishtiak

Posted by Ishtiak on December 30, 2008 at 10:07 PM PST #

Hi Ishtiak
Good question.
You can build on top of Shoal any required LB functionality. A Java client that uses Shoal could take advantage of the notifications, health checking etc to build out LB functionality for failover purposes. Shoal does not provide LB functionality as LB typically has its own set of functionality such as stickiness to support.

Hth
Shreedhar

Posted by Shreedhar Ganapathy on January 04, 2009 at 10:17 AM PST #

Unable to find the document, can anyone mail the document or share it

http://wikis.sun.com/display/shoal/Shoal+How-To+-+Configuring+for+cross+subnet+support

Posted by guest on January 14, 2012 at 09:37 PM PST #

Hi
Thanks for asking about this.
It appears that during a move of Sun wiki pages to Oracle systems, this page was lost.
However, the feature you are looking for on that page did not get supported at the time of writing of this blog entry.

In the upcoming GlassFish 3.1.2 release, cross subnet or non-multicast support is now added.
Could you post a question to the following mailing list to get instructions on setup ?
users at shoal dot java dot net

Thanks
Shreedhar

Posted by Shreedhar on January 15, 2012 at 01:59 AM PST #

Thanks for this insight to this framework...
Is there any support for management console either using JMX
or any other technology/framework?
In my project we need to have a GUI based Administrator console for
managing all the instances those are running using Shoal.

Posted by istek on January 17, 2012 at 12:13 AM PST #

Hi We have built the management and admin console support for shoal instances through GlassFish product/project - these are not independently available for only Shoal.
Are you using an application server cluster? In such a case you may want to use GlassFish which comes with Shoal group management as well as session persistence which are basic HA related requisites for any project with clustering requirements.

I'd encourage you to post your questions to users at shoal dot dev dot java.net mailing list so someone in addition to can respond to your questions.

hth
Shreedhar

Posted by Shreedhar on January 17, 2012 at 02:24 AM PST #

Hi Shreedhar,

I am doing a POC on creating a fault tolerant reliable, data replication middleware. I chose Shoal over Akka, for inter JVM message passing. While I am at a very preliminary level, I faced the following issue.

I have a master agent that will spawn slaves (separate JVM) on demand, and coordinate the task by notifications. For generating slaves, I am doing a Runtime.exec to create a new Java process. I do see the processes are getting creating, and on shutdown of the master the slaves are also brought down (through a shutdown hook). However, no signal passing is being observed (message/joining anything). I am testing this through Eclipse 3.7 on a 32 bit Windows Vista.

Am I missing something fundamental here? Since I have not much output to show, I have not posted anything on the mailing list. Can you please advise.

Regds,
Sutanu

Posted by guest on July 12, 2012 at 11:32 AM PDT #

Hi Sutanu
Do the spawning master and the spawned processes have code for registering to receive notifications? And check to see if they are starting a gms module with a common multicast address.
This sort of question is perfectly suited for the users alias in the shoal project.
Regards
Shreedhar

Posted by Shreedhar on July 12, 2012 at 11:45 AM PDT #

Hi Shreedhar,

Thanks for your comments. Has the mailing list ID changed? I posted a question to users@shoal.dev.java.net however got a failed delivery.

Regds,
Sutanu

Posted by Sutanu on August 09, 2012 at 08:37 AM PDT #

Hi Sutanu
Yes the mailing list id changed over a year ago - the dev part of the address is gone. Its users AT shoal.java.net
http://java.net/projects/shoal/lists/
hth
Shreedhar

Posted by Shreedhar on August 09, 2012 at 09:01 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Shreedhar Ganapathy

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today