A basic need for any scaling application. Data sharing across multiple JVMs is one way an application can be scaled. Imagine an application that is loaded once, and can be run across multiple JVMs which might be local or remote.
This can be achieved by the Master-Slave pattern bound by a common shared space.
Problem1: There must be some way to share the data amongst these processes in this space without using Queues or RMI.
Problem 2: Dividing the tasks into chunks to be taken up in the multiple JVMs and collating the result into a consistent single output which can be efficiently transported to intended recipients that may be single or multiple.
Problem 3: Allowing any arbitrary JVM to leave or become part of the shared space at any point of time without corrupting computational results.
There are tons of ongoing research in this regards and one that looks most promising is The Java(TM) Multi-Tasking Virtual Machine.
However, I believe there can be a way to create such a system without reverting to a new JVM or creating extensions thereof.
Comments (3)
Have you tried Terracota? It does exactly what you want.
http://www.terracotta.org
Fernando, thanks for the suggestion.
Yes I am aware of Terracotta and the DSM it provides. However, Terracotta uses Point to Point Connections, which I do not consider appropriate for the problem at hand. Further, I am not looking at a hub and spoke model that Terracotta presents.
Sandeep Phukan
Posted by Fernando Takai | December 8, 2008 12:10 PM
Posted on December 8, 2008 12:10
Sandeep -
Since you work for Oracle, perhaps you have heard of Coherence? It is the leading in-memory data grid product (in use by over 600 banks), and addresses all the needs you described.
Peace,
Cameron Purdy | Oracle
http://www.oracle.com/technology/products/coherence/index.html
Hi Purdy,
Thanks for your inputs. I have heard about Coherence. However I am looking at an Open Source solution that is not based on a point to point Connection.
Cheers!
Sandeep Phukan
Posted by Cameron Purdy | December 10, 2008 1:09 PM
Posted on December 10, 2008 13:09
Sandeep -
What do you mean by "point to point" connection?
Peace,
Cameron.
Hi Cameron,
By Point-To-Point I meant traditional networking protocol like TCP. I just found out that Coherence also supports TCMP, which uses multicasting for inter node communication.However I wasnt able to find much on TCMP itself. I would appreciate if you can send me some more information on Coherence TCMP that would enable me to evaluate it.
Thanks for the interest.
Cheers!
Sandeep Phukan
Posted by Cameron Purdy | December 12, 2008 2:04 PM
Posted on December 12, 2008 14:04