Wednesday Jun 06, 2012

Boost your infrastructure with Coherence into the Cloud

Authors: Nino Guarnacci & Francesco Scarano,  at this URL could be found the original article:

Thinking about the enterprise cloud, come to mind many possible configurations and new opportunities in enterprise environments. Various customers needs that serve as guides to this new trend are often very different, but almost always united by two main objectives:

  • Elasticity of infrastructure both Hardware and Software
  • Investments related to the progressive needs of the current infrastructure

Characteristics of innovation and economy.

A concrete use case that I worked on recently demanded the fulfillment of two basic requirements of economy and innovation.
The client had the need to manage a variety of data cache, which can process complex queries and parallel computational operations, maintaining the caches in a consistent state on different server instances, on which the application was installed.
In addition, the customer was looking for a solution that would allow him to manage the likely situations in load peak during certain times of the year.
For this reason, the customer requires a replication site, on which convey part of the requests during periods of peak; the desire was, however, to prevent the immobilization of investments in owned hardware-software architectures; so, to respond to this need, it was requested to seek a solution based on Cloud technologies and architectures already offered by the market.

Coherence can already now address the requirements of large cache between different nodes in the cluster, providing further technology to search and parallel computing, with the simultaneous use of all hardware infrastructure resources. Moreover, thanks to the functionality of "Push Replication", which can replicate and update the information contained in the cache, even to a site hosted in the cloud, it is satisfied the need to make resilient infrastructure that can be based also on nodes temporarily housed in the Cloud architectures.

There are different types of configurations that can be realized using the functionality "Push-Replication" of Coherence.

Configurations can be either:

  • Active - Passive
    •  Hub and Spoke
  • Active - Active
    • Multi Master
    • Centralized Replication

Whereas the architecture of this particular project consists of two sites (Site 1 and Site Cloud), between which only Site 1 is enabled to write into the cache, it was decided to adopt an Active-Passive Configuration type (Hub and Spoke). If, however, the requirement should change over time, it will be particularly easy to change this configuration in an Active-Active configuration type.

Although very simple, the small sample in this post, inspired by the specific project is effective, to better understand the features and capabilities of Coherence and its configurations.

Let's create two distinct coherence cluster, located at miles apart, on two different domain contexts, one of them "hosted" at home (on-premise) and the other one hosted by any cloud provider on the network (or just the same laptop to test it :)). These two clusters, which we call Site 1 and Site Cloud, will contain the necessary information, so a simple client can insert data only into the Site 1. On both sites will be subscribed a listener, who listens to the variations of specific objects within the various caches.

To implement these features, you need 4 simple classes:

Represents the POJO class that will be inserted into the cache, and fulfills the task of containing useful information about the hypothetical links navigation

Represents a link simulator, which has the task of randomly creating objects of type CachedResponse that will be added into the caches

Represents the model of our example, because it is responsible for receiving instructions from the controller and performing basic operations against the cache, such as insert, delete, update, listening, objects within the cache

It is our controller, which give commands to be executed within the cache of the two Sites

So, summarily, we execute the java class "Shell", asking it to put into the cache 100 objects of type "CachedResponse" through the java class "CacheCommands", then the simulator "ResponseSimulatorHelper" will randomly create new instances of objects "CachedResponse ". Finally, the Shell class will listen to for events occurring within the cache on the Site Cloud, while insertions and deletions are performed on Site 1.

Now, we realize the two configurations of two respective sites / cluster: Site 1 and Site Cloud.

For the Site 1 we define a cache of type "distributed" with features of "read and write", using the cache class store for the "push replication", a functionality offered by the project "incubator" of Oracle Coherence.

For the "Site Cloud" we expect even the definition of “distributed” cache type with tcp proxy feature enabled, so it can receive updates from Site 1.

For two clients "Shell" which will connect respectively to the two clusters we have provided two easy access configurations.

Now, we just have to get everything and run our tests.

To start at least one "storage" node (which holds the data) for the "Cloud Site", we can run the standard class  provided OOTB by Oracle Coherence

with the following parameters and values:

To start at least one "storage" node (which holds the data) for the "Site 1", we can perform again the standard class provided by Coherence

with the following parameters and values:

Then, we start the first client "Shell" for the "Cloud Site", launching the java class


using these parameters and values:


Finally, we start the second client "Shell" for the "Site 1", re-launching a new instance of class 


using  the following parameters and values:


 And now, let’s execute some tests to validate and better understand our configuration.

The purpose of this test is to load the objects into the "Site 1" cache and seeing how many objects are cached on the "Site Cloud".

  • Within the "Shell" launched with parameters to access the "Site 1", let’s write and run the command:
    • load test/100
  • Within the "Shell" launched with parameters to access the "Site Cloud" let’s write and run the command:
    • size passive-cache

Expected result

If all is OK, the first "Shell" has uploaded 100 objects into a cache named "test"; consequently the "push-replication" functionality has updated the "Site Cloud" by sending the 100 objects to the second cluster where they will have been posted into a respective cache, which we named "passive-cache".

The purpose of this test is to listen to deleting and adding events happening on the "Site 1" and that are replicated within the cache on "Cloud Site".

  • In the "Shell" launched with parameters to access the "Site Cloud" let’s write and run the command:
    • listen passive-cache/name like '%'
  • or
    • a "cohql" query, with your preferred parameters
  • In the "Shell" launched with parameters to access the "Site 1" let’s write and run the following commands:
    • load test/10
    • load test2/20
    • delete test/50

Expected result

If all is OK, the "Shell" to Site Cloud let us to listen to all the add and delete events within the cache "cache-passive", whose objects satisfy the query condition "name like '%' " (ie, every objects in the cache; you could change the tests and create different queries).
Through the Shell to "Site 1" we launched the commands to add and to delete objects on different caches (test and test2).
With the "Shell" running on "Site Cloud" we got the evidence (displayed or printed, or in a log file) that its cache has been filled with events and related objects generated by commands executed from the" Shell "on" Site 1 ", thanks to "push-replication" feature. 

Other tests can be performed, such as, for example, the subscription to the events on the "Site 1" too, using different "cohql" queries, changing the cache configuration,  to effectively demonstrate both the potentiality and  the versatility produced by these different configurations, even in the cloud, as in our case.

More information on how to configure Coherence "Push Replication" can be found in the Oracle Coherence Incubator project documentation at the following link:

More information on Oracle Coherence "In Memory Data Grid" can be found at the following link:

To download and execute the whole sources and configurations of the example explained in the above post,  click here to download them; After download the last available version of the Push-Replication Pattern library implementation from the Oracle Coherence Incubator site, and download also the related and required version of Oracle Coherence.

For simplicity the required .jarS to execute the example (that can be found into the Push-Replication-Pattern  download and Coherence Distribution download) are:

  • activemq-core-5.3.1.jar
  • activemq-protobuf-1.0.jar
  • aopalliance-1.0.jar
  • coherence-commandpattern-
  • coherence-common-
  • coherence-eventdistributionpattern-
  • coherence-functorpattern-
  • coherence-messagingpattern-
  • coherence-processingpattern-
  • coherence-pushreplicationpattern-
  • coherence-rest.jar
  • coherence.jar
  • commons-logging-1.1.jar
  • commons-logging-api-1.1.jar
  • commons-net-2.0.jar
  • geronimo-j2ee-management_1.0_spec-1.0.jar
  • geronimo-jms_1.1_spec-1.1.1.jar
  • http.jar
  • jackson-all-1.8.1.jar
  • je.jar
  • jersey-core-1.8.jar
  • jersey-json-1.8.jar
  • jersey-server-1.8.jar
  • jl1.0.jar
  • kahadb-5.3.1.jar
  • miglayout-3.6.3.jar
  • org.osgi.core-4.1.0.jar
  • spring-beans-2.5.6.jar
  • spring-context-2.5.6.jar
  • spring-core-2.5.6.jar
  • spring-osgi-core-1.2.1.jar
  • spring-osgi-io-1.2.1.jar

At this URL could be found the original article:

Authors: Nino Guarnacci & Francesco Scarano

Thursday Nov 17, 2011

Oracle Coherence & Oracle Service Bus: REST API Integration

This post aims to highlight one of the features found in Oracle Coherence which allows it to be easily added and integrated inside a wider variety of projects.  The features in question are the REST API exposed by the Coherence nodes, with which you can interact in the wider mode in memory data grid.

Oracle Coherence and Oracle Service Bus are natively integrated through a feature found in the Oracle Service Bus, which allows you to use the coherence grid cache during the configuration phase of a business service. This feature allows you to use an intermediate layer of cache to retrieve the answers from previous invocations of the same service, without necessarily having to invoke the real business service again. Directly from the web console of Oracle Service Bus, you can decide the policies of eviction of the objects / answers and define the discriminating parameters that identify their uniqueness.

The coherence REST APIs, however, allow you to integrate both products for other necessities enabling realization of new architectures design.  Consider coherence’s node as a simple service which interoperates through the stardard services and in particular REST (with JSON and XML). Thinking of coherence as a company’s shared service, able to have an implementation of a centralized “map and reduce” which you can access  by a huge variety of protocols (transport and envelopes).

An amazing step forward for those who still imagine connectors and code. This type of integration does not require writing custom code or complex implementation to be self-supported. The added value is made unique by the incredible value of both products independently, and still more out of their simple and robust integration.

As already mentioned this scenario discovers a hidden new door behind the columns of these two products. The door leads to new ideas and perspectives for enterprise architectures that increasingly wink to next-generation applications: simple and dynamic, perhaps towards the mobile and web 2.0.

Below, a small and simple demo useful to demonstrate how easily is to integrate these two products using the Coherence REST API. This demo is also intended to imagine new enterprise architectures using this approach.
The idea is to create a centralized system of alerting, fed easily from any company’s application, regardless of the technology with which they were built . Then use a representation standard protocol: RSS, using a service exposed by the service bus; So you can browse and search only the alerts that you are interested on, by category, author, title, date, etc etc..

The steps needed to implement this system are very simple and very few. Here they are listed below and described to be easily replicated within your environment. I would remind you that the demo is only meant to demonstrate how easily is to integrate Oracle Coherence and the Oracle Service Bus, and stimulate your imagination to new technological approaches.

1) Install the two products: In this demo used (if necessary, consult the installation guides of 2 products)

 - Oracle Service Bus ver.
 - Oracle Coherence ver. 3.7.1

2) Because you choose to create a centralized alerting system, we need to define a structure type containing some alerting attributes useful to preserve and organize the information of the various alerts sent by the different applications. Here, then it was built a java class named Alert containing the canonical properties of an alarm information:
- Title
- Description
- System
- Time
- Severity

3) Therefore, we need to create two configuration files for the coherence node, in order to save the Alert objects within the grid, through the rest/http protocol (more than the native API for Java, C + +, C,. Net). Here are the two minimal configuration files for Coherence:



This minimum configuration allows me to use a distributed cache named "alerts" that can  also be accessed via http - rest on the host "localhost" over port "8080", objects are of type “oracle.cohsb.Alert”.

4) Below  a simple Java class that represents the type of alert messages:

5) At this point we just need to startup our coherence node, able to listen on http protocol to manage the “alerts” cache, which will receive incoming XML or JSON objects of type Alert. Remember to include in the classpath of the coherence node, the Alert java class and the following coherence libraries and configuration files:

 At this point, just run the coherence class node “

advising you to set the following parameters:

6) Let's create a procedure to test our configuration of Coherence and in order to insert some custom alerts in our cache. The technology with which you want to achieve this functionality is fully not considerable: Javascript, Python, Ruby, Scala, C + +, Java.... Because the protocol to communicate with Coherence is simply HTTP / JSON or XML.
For this little demo i choose Java:

A method to send/put the alert to the cache:

A method to query and view the content of the cache:

Finally the main method that execute our methods:

 No special library added in the classpath for our class (json struct static defined), when it will be executed, it asks some information such as title, description,... in order to compose and send an alert to the cache and then it will perform an inquiry, to the same cache. At this point, a good exercise at this point, may be to create the same procedure using other technologies, such as a simple html page containing some JavaScript code, and then using Python, Ruby, and so on.

7) Now we are ready to start configuring the Oracle Service Bus in order to integrate the two products. First integrate the internal alerting system of Oracle Service Bus with our centralized alerting system based on coherence node. This ensures that by monitoring, or directly from within our Proxy Message Flow, we can throw alerts and save them directly into the Coherence node. To do this I choose to use the jms technology, natively present inside the Oracle Weblogic / Service Bus. Access to the Oracle WebLogic Administration console and create and configure a new JMS connection factory and a new jms destination (queue).

Now we should create a new resource of type “alert destination” within our Oracle Service Bus project. The new “alert destination” resource should be configured using the newly created connection factory jms and jms destination.

Finally, in order to withdraw the message alert enqueued in our JMS destination and send it to our coherence node, we just need to create a new business service and proxy service within our Oracle Service Bus project.
Our business service is responsible for sending a message to our REST service Coherence using as a method action: PUT

Finally our proxy service have to collect all messages enqueued on the destination, execute an xquery transformation on those messages  in order to translate them into valid XML / alert objects useful to be sent to our coherence service, through the newly created business service.

The message flow pipeline containing the xquery transformation:

Incredibly,  we just did a basic first integration between the native alerting system of Oracle Service Bus and our centralized alerting system by simply configuring our coherence node without developing anything.
It's time to test it out. To do this I create a proxy service able to generate an alert using our "alert destination", whenever the proxy is invoked.

After some invocation to our proxy that generates fake alerts, we could open an Internet browser and type the URL  http://localhost: 8080/alerts/  so we could see what has been inserted within the coherence node.

8) We are ready for the final step.  We would create a new message flow, that can be used to search and display the results in standard mode. To do this I choosen the standard representation of RSS, to display a formatted result on a huge variety of devices such as readers for the iPhone and Android. The inquiry may be defined already at the time of the request able to return only feed / items related to our needs.
To do this we need to create a new business service, a new proxy service, and finally a new XQuery Transformation to take care of translating the collection of alerts that will be return from our coherence node in a nicely formatted RSS standard document.

So we start right from this resource (xquery), which has the task of transforming a collection of alerts / xml returned from the node coherence in a type well-formatted feed RSS 2.0

our new business service that will search the alerts on our coherence node using the Rest API.

And finally, our last resource, the proxy service that will be exposed as an RSS / feeds to various mobile devices and traditional web readers, in which we will intercept any search query, and transform the result returned by the business service in an RSS feed 2.0.

The message flow with the transformation phase (Alert TO Feed Items):

Finally some little tricks to follow during the routing to the business service, - check for any queries present in the url to require a subset of alerts  - the http header "Accept" to help get an answer XML instead of JSON:

In our little demo we also static added some coherence parameters to the request:
I would like to get from Coherence that the results will be sorted by date, and starting from 1 up to a maximum of 100.

Just incredible, our centralized alerting system is ready.

Inheriting all the qualities and capabilities of the two products involved Oracle Coherence & Oracle Service Bus:
- RASP (Reliability, Availability, Scalability, Performance)

Now try to use your mobile device, or a normal Internet browser by accessing the RSS just published:

Some urls you may test:

Search for the last 100 alerts :

Search for alerts that do not have time set to null (time is not null):

Search for alerts that the system property is “Web Browser” (system = ‘Web Browser’):

Search for alerts that the system property is “Web Browser” and the severity property is “Fatal” and the title property contain the word “Javascript”  (system = ‘Web Broser’ and severity = ‘Fatal’ and title like ‘%Javascript%’)

To compose more complex queries about your need I would suggest you to read the chapter in the coherence documentation inherent the Cohl language (Coherence Query Language) .

Some useful links:

- Oracle Coherence REST API Documentation

- Oracle Service Bus Documentation

- REST explanation from Wikipedia

At this URL could be downloaded the whole materials of this demo

Author: Nino Guarnacci.


.. useful blog on java and middleware technologies.


« October 2016