Making Software Work

Coherence already provides HTTP Session caching support for ASP.NET and J2EE applications without any application changes. Coherence for .NET includes a custom SessionStateStoreProvider implementation that uses a Coherence cache to store session state. This makes Coherence for .NET the best solution for any large ASP.NET application running within a web farm. The HTTP session state of J2EE Web Applications is managed using Coherence*Web. Coherence*Web is an HTTP session management module dedicated to managing session state in clustered environments. Built on top of Oracle Coherence, it has a wide range of features and supports a wide range of J2EE containers. With WebLogic Suite Coherence*Web is built in, but for other containers a Coherence utility modifies the necessary settings in the Web Applications web.xml file to transfer management of the HTTP Session state from the container to Coherence. With PHP there is no “out-of-the-box” Coherence support for HTTP Session state management, until now. The rest of this article outlines how using a standard PHP extension Coherence can be used as a clustered, reliable HTTP session handler for PHP.

Like ASP.NET and J2EE application containers, PHP makes it quite easy to plug-in your own HTTP session handler and lots of different flavours exist already, from in-memory stores using tools like memcached to file and database persistent session mechanisms using products like MySQL.  Coherence is an ideal persistence store for HTTP sessions because it is:

• Very fast, enabling dynamic web pages to be quickly displayed
• Resilient and fault-tolerant, ensuring that process or server failures do not impact users – logging them off etc.
• Scalable, to support and ever increasing number of online users/customers
• Simple to integrate and manage, so the overhead of introducing Coherence does not burden either web development of operational staff

So can Coherence be used with PHP?. Well using the native Coherence C++ client to create a PHP extension that can then be wrapped using a PHP session_set_save_handler to setup some user defined session storage functions. A diagram to illustrate the relationship between the PHP runtime, Coherence C++ extension and the Coherence cache is shown below:

To include the PHP custom session handlers in your PHP page add the following to the top of your PHP page:

<?php
require 'coherence_session.php';
$s = new CoherencePHPSessionHandler();
.
.
?>

This basically enables a new session to be started using the custom session save handler. The custom session save handler looks like this:

<?php
class CoherencePHPSessionHandler
{
    private static $debug = False;

    /**
     * A reference to a Coherence named cache
     * @var resource
     */
    private $cache;

    /**
     * Session lifetime
     * @var resource
     */
    private $lifeTime;

    private static function Log($msg)
    {
      if(CoherencePHPSessionHandler::$debug)
      {
        echo "<br>DEBUG PHP: " . $msg . "\n";
      }
    }

    function __construct()
    {
        // get session-lifetime
        $this->lifeTime = get_cfg_var("session.gc_maxlifetime");

        CoherencePHPSessionHandler::Log("Lifetime: " . $this->lifeTime);
        $this->cache = new Cache("dist-php-sessions");

        if ($this->cache == null)
        {
            return false;
        }

        CoherencePHPSessionHandler::Log("Got cache handle");

        session_set_save_handler(array(&$this, 'open'),
                                array(&$this, 'close'),
                                array(&$this, 'read'),
                                array(&$this, 'write'),
                                array(&$this, 'destroy'),
                                array(&$this, 'gc'));
        register_shutdown_function('session_write_close');
        session_start();

        CoherencePHPSessionHandler::Log("Started session");
        return true;
    }

    /**
     * Open the session
     * @return bool
     */
    function open($savePath, $sessName)
    {
        // get session-lifetime
        $this->lifeTime = get_cfg_var("session.gc_maxlifetime");
        CoherencePHPSessionHandler::Log("Opening session");
        return true;
    }

    /**
     * Close the session
     * @return bool
     */
    public function close()
    {
        // ToDo: need to release Coherence resources
        return true;
    }

    /**
     * Read the session
     * @param int session id
     * @return string string of the session
     */
    public function read($id)
    {
        $result = $this->cache->get($id);

        if($result != null)
        {
          CoherencePHPSessionHandler::Log("Read session, id: " . $id . ", value: " . $result);
          return $result;
        }
        return '';
    }

    /**
     * Write the session
     * @param int session id
     * @param string data of the session
     */
    public function write($id, $data)
    {
        CoherencePHPSessionHandler::Log("Session value: " . serialize($data));

        $this->cache->put($id, $data, $this->lifeTime);
        CoherencePHPSessionHandler::Log("Written session, id: " . $id . ", value: " . $data);
        return true;
    }

    /**
     * Destoroy the session
     * @param int session id
     * @return bool
     */
    public function destroy($id)
    {
        $this->cache->remove($id);
        return true;
    }

    /**
     * Garbage Collector
     * @param int life time (sec.)
     * @return bool
     * @see session.gc_divisor      100
     * @see session.gc_maxlifetime 1440
     * @see session.gc_probability    1
     * @usage execution rate 1/100
     *        (session.gc_probability/session.gc_divisor)
     */
    public function gc($max)
    {
        return true;
    }
}
?>

As for the Coherence PHP extension there is too much to show here, though in total it does not amount to a lot of code. The C++ client class that wraps the Coherence client API is also very simple and the class functions are shown below:

#include "cache.h"

extern "C" {
#include <stdio.h>
#include <stdlib.h>
#include "php.h"
}

static char * copyString(const char *str)
{
    char *s;

    if(str != NULL && (s = (char *)emalloc(sizeof(char) * (strlen(str)  + 1))) == NULL)
    {
        return NULL;
    }
    else
    {
        if(strncpy(s, str, strlen(str) + 1) == NULL)
        {
            efree(s);
            return NULL;
        }
        return s;
    }
}

Cache::Cache(const char *name)
{
    // Get handle to a Named Cache
    String::View vsCacheName = String::create(name);
    hCache = CacheFactory::getCache(vsCacheName);
}

const char *Cache::getCacheName()
{
    String::View name = cast<String::View>(hCache->getCacheName());

    return copyString(name->getCString());
}

char* Cache::put(const char *key, const char *value, long ttl)
{
    if(key == NULL || value == NULL)
    {
        throw std::invalid_argument("Key or value for put() NULL");
    }
    String::View vKey = String::create(key);
    String::View vValue = String::create(value);
    String::View vOldValue = cast<String::View>(hCache->put(vKey, vValue, ttl));

    return (vOldValue == NULL ? NULL : copyString(vOldValue->getCString()));
}

char* Cache::remove(const char *key)
{
    String::View vKey = String::create(key);
    String::View vOldValue = cast<String::View>(hCache->remove(vKey));

    return (vOldValue == NULL ? NULL : copyString(vOldValue->getCString()));
}

char* Cache::get(const char *key)
{
    String::View vKey = String::create(key);
    String::View vOldValue = cast<String::View>(hCache->get(vKey));

    return (vOldValue == NULL ? NULL : copyString(vOldValue->getCString()));
}

int Cache::size()
{
    return (int)hCache->size();
}

The Coherence PHP extension can also be used for simply caching PHP values, as well as PHP session information. The above example was tested on Oracle Enterprise Linux 4, but can easily be compiled on Windows. You can download the whole example from here to try it out for yourself. Instructions are included which should hopefully be easy to follow. A couple of outstanding issues still remain with the Coherence PHP extension which I have not been able to resolve. Although simple PHP types as well as objects and arrays can be cached – both as keys and values – I could not get objects with private or protected variables to serialize. If anyone more experienced in the internals of PHP can help here it would be much appreciated, as I drew a blank.

With release 3.5 of Oracle Coherence it is now possible to query, aggregate and modify serialized POF (Portable Object Format) values stored in a cache natively, that is without writing a Java representation of the object. So  .NET and C++ developers can just write C# etc. and C++.

This is achieved with the introduction of POF extractors and POF updaters, that fetch and update native POF objects without de-serializing the value to Java. As well as allowing .NET and C++ developers to just work in the language they are most comfortable and productive in, this new feature also has a number of other benefits:

• It dramatically improves performance, as no de-serialization needs to take place, so no new objects need to be created – or garbage collected.
• Less memory is required as a result.
• The development and deployment process is simpler, as no corresponding Java classes need to be created, managed and deployed.

However, there are some occasions where you do still need to create complementary Java objects to match your .NET or C++ objects. These are:

• When you want to use Key Association, as Coherence will always de-serializes keys to determine whether they implement KeyAssociation.
• If you use a Cache Stores - Coherence passes the de-serialized version of the key and value to the cache store to write to the back end so that ORM (Object Relational Mapping) tools, like Hibernate and EclipseLink (JPA) have access to the Java objects.

To update a value in a cache from C# you would use code similar to that shown below:

    // Parameters: key, (extractor, new value)
    cache.Invoke(0, new UpdaterProcessor(new PofUpdater(BetSerializer.ODDS), 1)); 

Here a cache value identified by the key ‘0’ is being updated using an UpdateProcessor. The ValueUpdator being used is a C# PofUpdater which will access the attribute at offset BetSerializer.ODD in the serialized POF value. Values held in a cache are held in a serialized format. When the POF serialization mechanism is used the values of an object are written and read from the POF stream in the same order. So here a constant  (BetSerializer.ODDS) is being used to provide a more readable representation for an offset like 3, i.e. the 3rd object value to be written and read from the POF stream.

For reading values from a cache in C# a similar approach is used, as shown below:

    // Query cache for all entries whose event name is 'FA Cup'
    EqualsFilter equalsFilter =
        new EqualsFilter(new PofExtractor(BetSerializer.EVENT_NAME), "FA Cup");

    Object[] results = cache.GetValues(equalsFilter);
    Console.WriteLine("Filter results:");

    for (int i = 0; i < results.Length; i++)
    {
        Console.WriteLine("Value = " + results[i]);
    }

Here all cache values that have an event name or “FA Cup” will be returned by the EqualsFilter. The C# PofExtractor is being used to extract the attribute at the offset BetSerializer.EVENT_NAME, which will be a number , like 4 indicating it was the 4th value to be written and read from the POF stream when the object it was part of was serialized.

This is a great new feature of Coherence and very easy to use. If you would like to see the full example the above extracts were taken from the you can download it from here. Happy coding - in C# and C++ ;).

A common requirement when using Coherence is for the data in it to remain synchronised with database. If all changes to the database flow through Coherence or the cached data can be periodically refreshed (using the refresh-ahead mechanism) then Coherence will be aware of of any database changes. However, if another application changes the database outside of Coherence or the database changes need to be relayed to Coherence in real-time then an alternative approach is required.

To push database changes to Coherence in real-time when they are being made outside of Coherence, a queue needs to be used. Fortunately most databases support the queuing of transactional change information, either implicitly or explicitly. In this example architecture the explicit queuing of changes to an Oracle database, using Triggers and Oracle’s Advanced Queuing (AQ) technology, will be used to propagate them to a Coherence cache holding objects representing the same information.

Overview

Below is a diagram showing how the changes flow from the Oracle database to the Coherence cache.

In the example the read/receive operation from the queue is performed as a transaction, so the message is only removed from the queue once the corresponding cache object has been updated. This ensures that if the Coherence JVM/node that the client runs on fails the update message will not be lost. Although the example code focuses on cache updates it could easily be modified to accommodate inserts and deletes as well.

To enable Advanced Queuing (AQ) to be used for propagating table changes a number of database permissions first need to be granted. The PL/SQL to do this is shown below:

-- Execute as sys/welcome1

GRANT SELECT_CATALOG_ROLE TO scott;
GRANT EXECUTE ON DBMS_APPLY_ADM TO scott;
GRANT EXECUTE ON DBMS_AQ TO scott;
GRANT EXECUTE ON DBMS_AQADM TO scott;
GRANT EXECUTE ON DBMS_CAPTURE_ADM TO scott;
GRANT EXECUTE ON DBMS_FLASHBACK TO scott;
GRANT EXECUTE ON DBMS_STREAMS_ADM TO scott;
EXECUTE dbms_aqadm.grant_system_privilege('ENQUEUE_ANY', 'scott', TRUE);
EXECUTE dbms_aqadm.grant_system_privilege('DEQUEUE_ANY', 'scott', TRUE);
GRANT aq_administrator_role TO scott;
GRANT EXECUTE ON dbms_lock TO scott;
GRANT EXECUTE ON sys.dbms_aqin TO scott;
GRANT EXECUTE ON sys.dbms_aqjms TO scott;
EXIT;

Note: These GRANT statements need to be executed as the database administrator (sys) or another database user who has privileges to grant them.

Then the AQ queue need to be setup and the PL/SQL procedure and trigger created to put the database table changes in the appropriate queue. Below is the PL/SQL used to do this:

-- Execute as scott/tiger

-- Create queue
EXECUTE dbms_aqadm.stop_queue(queue_name => 'trade_queue');
EXECUTE dbms_aqadm.drop_queue(queue_name => 'trade_queue');
EXECUTE DBMS_AQADM.DROP_QUEUE_TABLE(queue_table => 'trade_queue_table');
EXECUTE dbms_aqadm.create_queue_table(queue_table => 'trade_queue_table',queue_payload_type => 'sys.aq$_jms_text_message', multiple_consumers => false);
EXECUTE dbms_aqadm.create_queue(queue_name => 'trade_queue', queue_table =>'trade_queue_table', queue_type => DBMS_AQADM.NORMAL_QUEUE, retention_time => 0, max_retries => 5, retry_delay => 60);
EXECUTE dbms_aqadm.start_queue(queue_name => 'trade_queue');

-- Create table

DROP TABLE Trade CASCADE CONSTRAINTS;
CREATE TABLE Trade
(
    Id     NUMBER PRIMARY KEY,
    Symbol VARCHAR2(5)       ,
    Created DATE             ,
    Quantity NUMBER          ,
    Amount   NUMBER(8,2)
);

-- Enable SERVEROUTPUT in SQL Command Line (SQL*Plus) to display output with
-- DBMS_OUTPUT.PUT_LINE, this enables SERVEROUTPUT for this SQL*Plus session only
SET SERVEROUTPUT ON
CREATE OR REPLACE
PROCEDURE testmessage(text_message VARCHAR2)
AS
  msg SYS.AQ$_JMS_TEXT_MESSAGE;
  msg_hdr SYS.AQ$_JMS_HEADER;
  msg_agent SYS.AQ$_AGENT;
  msg_proparray SYS.AQ$_JMS_USERPROPARRAY;
  msg_property SYS.AQ$_JMS_USERPROPERTY;
  queue_options DBMS_AQ.ENQUEUE_OPTIONS_T;
  msg_props DBMS_AQ.MESSAGE_PROPERTIES_T;
  msg_id RAW(16);
  dummy VARCHAR2(4000);
BEGIN
  msg_agent := SYS.AQ$_AGENT(' ', NULL, 0);
  msg_proparray := SYS.AQ$_JMS_USERPROPARRAY()  ;
  msg_proparray.EXTEND(1);
  msg_property := SYS.AQ$_JMS_USERPROPERTY('JMS_OracleDeliveryMode', 100, '2', NULL, 27);
  msg_proparray(1) := msg_property;
  msg_hdr := SYS.AQ$_JMS_HEADER(msg_agent,NULL,'<USERNAME>',NULL,NULL,NULL,msg_proparray);
  msg := SYS.AQ$_JMS_TEXT_MESSAGE(msg_hdr,NULL,NULL,NULL);
  msg.text_vc  := text_message;
  msg.text_len := LENGTH(msg.text_vc);
  DBMS_AQ.ENQUEUE(queue_name => 'trade_queue' , enqueue_options => queue_options , message_properties => msg_props , payload => msg , msgid => msg_id);
END;
/
CREATE OR REPLACE TRIGGER TradeAQTrigger AFTER INSERT OR UPDATE ON Trade
FOR EACH row DECLARE xml_complete VARCHAR2(1000);
BEGIN
    xml_complete := '<?xml version="1.0" encoding="UTF-8" ?>' ||
      '<TradeElement xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ' ||
      'xsi:schemaLocation="http://www.oracle.com/coherenceaq src/aq-object.xsd" ' ||
      'xmlns="http://www.oracle.com/coherenceaq">' ||
    '<Id>' || :new.ID || '</Id>' ||
    '<Symbol>' || :new.SYMBOL || '</Symbol>' ||
    '<Quantity>' || :new.QUANTITY || '</Quantity>' ||
    '<Amount>' || :new.AMOUNT || '</Amount>' ||
    '<Created>' || :new.CREATED || '</Created>' ||
    '</TradeElement>';   
    testmessage(xml_complete);
END;
/
SHOW ERRORS;

The Java code that runs in the background Coherence thread – started by the BackingMapListener – that reads messages from the queue looks like this:

.
.
.

/**
* <p>AQ client interface</p>
*/
public class AQClient
{
  .
  .
  .

  /**
   * Send an acknowledgement message for a received message. This enables the
   * message to be kept by the sender if client dies. An acknowledgement should only
   * be sent when the received message has been saved, e.g. in a cache
   *
   * @throws JMSException If an exception occurs sending an acknowledgement
   */
  public void aknowledgeMessage()
    throws JMSException
  {
    textMsg.acknowledge();
  }

  /**
   * Reads XML text message from the queue and transforms it into a Trade
   * object using JAXB classes.
   *
   * @return a Trade object
   * @throws JMSException If an exception occurs sending an acknowledgement
   * @throws JAXBException If an exception occurs during JAXB processing
   */
  public Trade receiveMessage()
    throws JMSException, JAXBException
  {
    // Wait for a message to show up in the queue
    textMsg = (TextMessage) queueReciever.receive();
    System.out.println("Message is: " + textMsg.getText());

    // Use JAXB to create object
    JAXBContext jc = JAXBContext.newInstance("com.oracle.demo.model");
    Unmarshaller u = jc.createUnmarshaller();
    JAXBElement<Trade> trade =
      (JAXBElement<Trade>) u.unmarshal(new StreamSource(new StringReader(textMsg.getText())));
    Trade t = trade.getValue();
    System.out.println("Trade created on: " + t.toString());
    return t;
  }
}

A full code example can be found here, including all the necessary PL/SQL scripts. All you need to try it out is Coherence, Oracle XE and Coherence .NET – if you want to see the changes propagated to a .NET Coherence client.

If changes in the Coherence cache data also need to be written back to the database then something like a status field or flag would need to be added to the the cached objects and table, to ensure that a circular loop isn’t created. For instance the following logic could enable bi-directional synchronization:

1. Propagate flag added to cached object (Trade) and database table (Trade) to signify if changes should be propagated by the after update trigger from the database to Coherence
2. Data changes propagated from Coherence to Database have propagate flag set to FALSE.
3. Introduce a new before insert/update trigger to reset the propagate flag to TRUE when it is FALSE.
4. When the after insert/update trigger fires only place the new value on the queue if the old value is not FALSE.

A modified example of the PL/SQL to add this functionality is in the setup-queue2.sql file, though you will need to add the additional column to the Trade table and attribute to the Trade object.

Summary

This approach is not the only mechanism for propagating changes from an Oracle Database to Coherence. Using a BackingMapListener to start the background queue client thread has an added benefit. If the node it is running on fails, Coherence will re-start the queue client on another node as part of its recovery process, i.e. when the queue configuration entry is failed-over and the new primary created it will cause a new background queue client thread to be started.

Alternative approaches to the one above might be to use Oracle Streams or the Data Change Notification mechanism in Oracle 11g

On my previous posting about using Excel as a Coherence client I talked about how this could be done but gave no example code. In this posting I will explain how to do this in more detail and provide the code to a working example that you can modify to use your own objects. In the example a .NET client loads Stock objects into a Coherence cache and then randomly updates the Stock prices. An Excel spreadsheet can then be opened that is a client for the Coherence cache, receiving changes to the Stock objects – the price changes. The prices can also be updated from the spreadsheet, so the communications is both ways.

Stock price updates are pushed down to the Excel spreadsheet using the standard RTD (Read Time Data) server mechanism. As for the cache updates from Excel, they are send through a Coherence COM interface. VBA User Defined Functions (UDF’s) provide the interface in Excel for specifying where event data goes or which cell updates get mapped onto cache objects. These are added to Excel via the Add-in option from the Tools menu.

The UDF’s also hide some of the complexity of the interfaces from users. Here are he 3 UDF’s that provide the interface to Coherence:

• For updating properties of Stock objects in the cache
• UpdateDoubleProperty("dist-stocks",E4,"ORCL","StockPrice")
• UpdateStringProperty("dist-stocks",E4,"ORCL","StockPrice")
• For receiving update events for Stock objects in the cache
• RealTimeData("dist-stocks","ORCL","StockPrice")

In the UDF’s above “dist-stocks” is the name of the Coherence cache, “ORCL” is the key for the object you wish to update or receive change events from (this must be a string in the example) and “StockPrice” is the name of the property you want to update or get the latest value of. “E4” in the UpdateDoubleProperty() is the cell to take the new property value from – when it changes.

If properties of objects in the cache that you wish to receive updates about or change are in nested objects you can specify the target property via a “.” or if the target is in an object nested in a List or array you can use the “[ ]” operator. So for instance if the cache contained a Portfolio object with nested Stock objects in an array, you could specify the target Stock price for one of the nested Stocks as “Stocks[2].StockPrice”. This would get the 3rd Stock in a property of the Portfolio object called “Stocks” and from that Stock the “StockPrice” property. At the moment the UDF functions only support String and Double properties – but they could easily be enhanced to support other properties. This property specification method can also be nested as much as you like and provide as convenient way of specifying a target property as a string in Excel.

The easiest way to see how this all works in practice is to download the example and try it out – there is a readme.txt file in the example that explains how to set it up and try it out. Some of the things you will need to run the example are:

• Coherence for Java and .NET 3.4.2. It should work with more recent versions, though this is the release I have tested the example against.
• Java JDK 1.4.2 (or above)
• Microsoft Visual Studio 2008 and .NET 3.5. I tried to build the example using Visual Studio Express and via the command line but was unable to specify that the Coherence configuration files be embedded resources. Also someone has successfully back-ported the example to Visual Studio 2005 (though I don’t have the code)
• Excel 2003. It should work with later versions, though I am not sure when Excel started supporting the RTD mechanism.
• Office XP Primary Interop Assemblies (for Office 2003). This is need for the Excel integration assemblies.

Have fun trying this out and if anyone has any additional thoughts, recommendations or feedback I’d be keen to hear them.

Although Coherence is often used to manage Servlet session state for J2EE applications – without changing the application code – it can also be used by J2EE Web applications to access data in a simple cache. This article discusses how to do this and how you should package your Web application so that it is a completely self contained Coherence client. Although the principles should work for all J2EE applications servers you will not be surprised to learn that I have only tested this on WebLogic Server (10.3).

The Web application outlined below is a Coherence*Extend Client, that means that it connects over a TCP/IP connection to the Coherence cluster (which uses UDP for inter-node communications) holding the cache data. A Proxy service run either on one or more of the cache nodes (or in a separate JVM) is used to pass requests and response between the client and the Coherence cluster – in a similar fashion to the way an Oracle Database listener works.

Note, the client could also be a Coherence Data Client, which means that the Web client would actually be part of the Coherence cluster. In this case the client would need to be ‘storage disabled’ through a system property or through the tangosol-coherence-override.xml configuration file. This means that although it will be part of the cluster it will not actually hold any data, so when the client is deployed or un-deployed there will be no cluster overhead re-organising cache data.

Accessing the configuration files

You really have a choice here. put them in the packaged Web application or reference them externally through the CLASSPATH or via a system property. Here I am going to package them with the Web application to make is completely self-contained. The code to do this is shown below:

 56    /**
 57     * <p>Reference to ConfigurableCacheFactory that can be used to
 58     * create a cache.</p>
 59     */
 60    private ConfigurableCacheFactory factory = null;
 61  
 62    /**
 63     * @inheritDoc
 64     */
 65    public void init(ServletConfig config)
 66      throws ServletException
 67    {
 68      super.init(config);
 69      // Create DefaultConfigurableCacheFactory using cache configuration file
 70      // in classes dir
 71      factory =
 72          new DefaultConfigurableCacheFactory("/config/cache-config.xml",
 73                                              getClass().getClassLoader());
 74    }

Here a DefaultConfigurableCacheFactory is used to read the configuration file used to setup the Coherence client parameters. All the configuration files that the Coherence client needs will be stored in the /WEB-INF/classes/config directory of the Web application. These are:

• cache-config.xml indicating where the Coherence cluster can be contacted
• pof-config.xml indicating the mapping between client and server objects. As both the client and server are Java only one object needs to be defined.
• tangosol-coherence-override.xml encapsulated many of the Coherence settings the are often specified as system properties.

The cache is accessed in the doGet(..) method of Servlet  as shown below:

 76    /**
 77     * @inheritDoc
 78     */
 79    public void doGet(HttpServletRequest request,
 80                      HttpServletResponse response)
 81      throws ServletException, IOException
 82    {
 83      // Create response
 84      response.setContentType(CONTENT_TYPE);
 85      PrintWriter out = response.getWriter();
 86      out.println("<html>");
 87      out.println("<head><title>CoherenceServlet</title></head>");
 88      out.println("<body>");
 89      // Get Coherence cache date
 90      NamedCache cache =
 91        factory.ensureCache("stock-cache", getClass().getClassLoader());
 92      // Get Stock object from cache
 93      Stock s = (Stock) cache.get("ORCL");
 94      double price = 0;
 95      if (s != null)
 96      {
 97        price = s.getPrice();
 98      }
 99      // Output stock price in response
100      out.println("<p>The latest Oracle Stock Price is: <b>" + price +
101                  "</b></p>");
102      out.println("</body></html>");
103      out.close();
104    }

The other source file that needs to be deployed with the Servlet is the Stock object that is being cached – a separate Java application is used in this scenario to add a Stock object to the “stock-cache” and then continually update it. This means that if the content displayed by the Servlet is refreshed in a browser a new stock price will appear.

Packaging it all up

To package it all up you naturally wrap the Servlet and Stock classes – along with the Coherence configuration files and supporting web configuration files – in a WAR file. This WAR file then needs to be wrapped in an EAR file with the coherence.jar file and a reference to this JAR in the MANIFEST.MF file as follows:

Class-Path: lib/coherence.jar

Note in the Eclipse project the coherence.jar file is in the base dir of the EAR file. The complete structure of the EAR file is shown below:

When the application is deployed and run you should see something like this which if you keep refreshing should change.

Summary

That’s it. You should now be able to create a cache, put some data into it and then access it from the above Servlet. The full source code for this example can be found here. There is a JDeveloper 11g project and an Eclipse Eurpoa project. There are some batch files to run-up a cache server and a separate client to insert and update Stock data, but unfortunately they are only in JDeveloper project – at the moment. See the readme.txt file for details of how to set things up.

Often Coherence is thought of as a Java data caching product, and the server side component – where the data is kept is. But the client can not only be a Java application, it can also be a native .NET or C++ application. The native C++ application can also run on Windows, Linux or Solaris. So getting back to .NET. Since the .NET client is just a native C# or VB.NET application (no Java) you can do some nice things, like use the Real Time Data (RTD) Server API to integrate it with Excel, turning Excel into a real-time Coherence client. If you want to see a flash demonstration then click here to view it in action.

The screen shot of the demonstration shows how a constantly updating graph (shown below) of stock price information can be displayed in Excel. The data which is constantly changing, in this case stock prices, is held in a Coherence cache. A separate client application is changing the stock prices and these changes in the data cache are then being pushed to the .NET Coherence client (RTD Server) and on to the Excel spreadsheet.

The RTD Server is a C# .NET client packaged as an .NET Assembly/DLL and then registered as a COM object. The example in the demonstration is pretty flexible but the next version will be enhanced to provide a great range of filtering.

Now the example being shown in the demonstration is a first cut (developed with help from Aleksandar Seovic). An improved version will be published very soon with a fuller explanation about it works along with the code itself.

What is Coherence and why would you used it?

Oracle Coherence stores data in-memory so access to it is very fast. Therefore, you would commonly use Coherence to provide fast access to data. Typically this is necessary in applications where performance and latency are key.

How does it work?

Oracle Coherence is a small Java programme that stores data in lookup tables, key/value pairs, or a Map of Java objects in memory (or a cache in Coherence terminology). Well I can do that myself I can hear you say, why do I need Coherence. Well if you do it yourself and crate a lookup table in your own Java program what happens if the Java program crashes?, all your data in-memory is lost. You could persist the data to disk or a database as soon as it changes, but then that would degrade performance. Another problem is what happens when you need to store more data than the Java program has access to (i.e. in its heap)?. Well you could increase the amount memory the Java program has available, but if you are running on a 32 Bit Operating System (OS) this will be limited to 2 GB and even if you are on a 64 Bit OS this can cause problems. Java programs which use large amounts of memory can suffer from un-predictable pauses, anything upwards from 1-2 seconds depending on the amount of memory used. This happens when the Java Virtual Machine (JVM) tidies up all the un-used data (or performs a full Garbage Collection (GC)). For applications that need to provide a consistent level of service, e.g. a customer facing web application, this is often unacceptable.

Oracle Coherence address both of these problems. When lots of Coherence Java programs (or cache servers) are started on one or more connected computers they 'glue' themselves together (to form a cluster) using mulit-cast (or a specified IP of addresses). The look-up table, or cache, can then be spread or partitioned across all the Coherence Java programs (nodes) in the cluster. More nodes can be started to increase the size of the cache ,you just have to start more nodes and they will automatically join the cluster – no re-start  is required. Since a cache can be built from lots of nodes with a relatively small memory footprint processing outages, when the JVM is performing memory housekeeping chores (GC's), is also greatly reduced as you can run lots of JVM’s with a small memory footprint and the effect of any particular JVM GC is also diluted across the whole cluster.

To address the problem of loosing data, if a node or server running a number of nodes crashes, Coherence can keep backup copies of data on different nodes and servers. All changes to the primary copy of a value are synchronously made to the backup, to make sure they are always the same, and if either are lost a new primary or backup is created from the remaining value. Furthermore, Coherence will automatically re-organise the partitioned data in a cache if a failure occurs to ensure that it is re-balanced across the cluster and any in-flight client transactions or queries will still complete successfully.

So simplistically Coherence is just a in-memory lookup table. The clever thing that Coherence does is enable the lookup table to grow and be resilient.

How do I use it?

Well that's enough background information. To use Coherence you only need the Java runtime (1.4+). You can download the Oracle JRockit Java runtime and Coherence from the Oracle Technology Network.

• Coherence can be downloaded from here. To install Coherence you just unzip it - that's it!.
• Oracle JRockit JVM can be downloaded from here (if you do not already have Java installed)

In addition to being able to put data in a lookup table and look it up Coherence also supports queries, aggregations, transactions, events and other features. Here we will just concentrate on putting data in a cache and looking it up - to keep things quick and simple.

Coherence works in a sort of client-server mode. I say sort of because a client will make a direct request to the node in the cluster which has the data it wants, rather than go through a central server node or lookup service. This also means that Coherence has no 'single point of failure' or 'single point of bottleneck'. Every node in a cluster is effectively equal and is just responsible for its own data.

So what does a Coherence client look like?

Well the object we are going to cache looks like this:

package com.oracle.coherence.demo;

import java.io.Serializable;

import java.util.Date;

/**
* Order class representing an order for a shares
*/
public class Order
  implements Serializable
{
  private String symbol;
  private int quantity;
  private double amount;
  private Date timeStamp;

  /**
   * Default constructor
   */
  public Order()
  {
  }

  /**
   * Constructor
   * @param symbol the Order symbol, e.g. ORCL
   * @param quantity the number of shares
   * @param amount the amount of the shares to buy at
   * @param timeStamp a time stamp
   */
  public Order(String symbol, int quantity, double amount, Date timeStamp)
  {
    this.symbol = symbol;
    this.quantity = quantity;
    this.amount = amount;
    this.timeStamp = timeStamp;
  }

  /**
   * Gets the symbol
   * @return the symbol
   */
  public String getSymbol()
  {
    return symbol;
  }

  /**
   * Sets the new symbol
   * @param symbol the new symbol value
   */
  public void setSymbol(String symbol)
  {
    this.symbol = symbol;
  }

  /**
   * Gets the quantity
   * @return the quantity
   */
  public int getQuantity()
  {
    return quantity;
  }

  /**
   * Sets the new quantity
   * @param quantity the new quantity
   */
  public void setQuantity(int quantity)
  {
    this.quantity = quantity;
  }

  /**
   * Gets the amount
   * @return the amount
   */
  public double getAmount()
  {
    return amount;
  }

  /**
   * Sets the amount
   * @param amount the new amount
   */
  public void setAmount(double amount)
  {
    this.amount = amount;
  }

  /**
   * Gets the time stamp
   * @return the time stamp
   */
  public Date getTimeStamp()
  {
    return timeStamp;
  }

  /**
   * Sets the new time stamp
   * @param timeStamp the new time stamp
   */
  public void setTimeStamp(Date timeStamp)
  {
    this.timeStamp = timeStamp;
  }
}

The Coherence client application looks like this:

package com.oracle.coherence.demo;

import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;

import java.util.Date;
import java.util.Iterator;
import java.util.Set;

/**
* Simple Coherence cache client example
*/
public class SimpleCoherenceClient
{
  /**
   * Main method
   * @param args not used
   */
  public static void main(String[] args)
  {
    // Get a handle/reference to a cache by name - which does not have to exist
    NamedCache namedCache = CacheFactory.getCache("test");

    // Create an object value to put in the cache
    Order value = new Order("ORCL", 400, 19.4, new Date());

    // Create a key for the object
    Integer key = new Integer(0);

    // Put an item into the cache
    namedCache.put(key, value);

    // Retrieve the object based upon its key
    Order order = (Order) namedCache.get(key);

    System.out.println("------------------------------------------------------");
    System.out.println("Retrieved order: " + "symbol=" +
                       order.getSymbol() + ", amount=" +
                       order.getAmount() + ", quantity=" +
                       order.getQuantity() + ", timestamp=" +
                       order.getTimeStamp());
    System.out.println("------------------------------------------------------");

    // Shutdown client connection to cache
    CacheFactory.shutdown();
  }
}

As you can see the client code to add and retrieve an object from a Coherence cache is trivial. The above example can be downloaded from here, with all the files ready to run it. Look at the readme.txt file for instructions.

When you run a Coherence client all you need to do is include the Coherence Jar files in the CLASSPATH and either include the XML configuration file in the CLASSPATH or indicate its location through a system property. That's it. Through multi-cast it will locate the cluster on the default TCP port. Using a specific IP address etc. for clustering is simply a matter of overriding default configuration file settings.

So how fast is Coherence?

Well that depends on how much data you need to access, what your network is like, how fast your hardware is etc, but it is usually run on commodity hardware (blades and GBit Ethernet). For more information on performance and scalability see the results of our internal test here.

How much data can Coherence store?

Some Coherence customers have caches with 100's of GB of data and 100's of nodes. Others only cache a few GB of data.

Can I save cache entries to a database?

Yes. Coherence provides the facility to persist cache data to any database (using JPA, Hibernate or Toplink) synchronously or asynchronously. Its also very easy to plug-in persistence to everything from a file system to a Mainframe.

I use .NET of C++ not Java. Can I still use Coherence?

Coherence also comes with native .NET and C++ client libraries. When using these technologies, .NET or C++ objects are created to map onto Java objects and Coherence then transforms the .NET or C++ objects to and from the Java objects when the clients communicate with the Java cache servers.

There is obviously a lot more to Coherence than has been covered here, but hopefully this has given you a flavour of how to use it. For more information about the other features mentioned, examples and white papers etc,go to the Coherence home page on Oracle Technology Network.