Hello, Coherence Community Edition: Creating cloud native stateful applications that scale, Part 1

Use the open source Oracle Coherence Community Edition to create stateful applications that are as easy to scale, if not easier, than the stateless applications you are building today.

August 14, 2020

Download a PDF of this article

Oracle Coherence is nearly 20 years old. It started as a distributed caching product and then evolved into an in-memory data grid. It’s an essential tool for improving the performance and scalability of Java EE applications, and it’s widely used for large-scale projects.

Think of it as a scalable, concurrent, fault-tolerant java.util.Map implementation that is partitioned across multiple JVMs, machines, and even data centers. That is a major oversimplification, of course, but it is sufficient for now.

For most of its history, Oracle Coherence was a commercial product with a fairly high price tag. That limited its appeal mainly to corporate users and took it out of consideration for many applications, including smaller or open source projects that didn’t absolutely need the features it offers.

All that changed on June 25, 2020, when Oracle released Coherence Community Edition (CE), an open source version of the product.

Coherence CE may often be the best option you have when building modern, cloud native applications and services. Along with my colleague Harvey Raja, I recently wrote A Gentle Introduction to Coherence, covering what the platform is and why you should consider using it for your next application, so I will not repeat that information here.

In this article, I will focus on how to build scalable stateful applications using Coherence CE.

Extending the example To Do List service application

In this article, I will extend the To Do List”service application used as a quick start example on the Coherence CE website (see Figure 1). I’m doing this for two reasons:

  • It has a very simple domain model that everyone understands, so this article can focus on the usage of Coherence CE.
  • Despite its simplicity, the sample application demonstrates many Coherence CE features from basic reads and writes, to queries and aggregations, to in-place processing and events.

Screenshot of the To Do List application

Figure 1. Screenshot of the To Do List application

I’ll make my application a lot more interesting than the quick start example, though: In addition to the Helidon REST service implemented there, my project will do the following:

  • Add support for server-sent events (SSEs) to the REST API to broadcast Coherence CE events to REST clients.
  • Implement a React-based web front end that will use the Helidon REST API, which will be served by the Helidon Web Server as well.
  • Configure Coherence CE’s gRPC server and implement a JavaFX front end that uses the native Java client to interact with Coherence CE over gRPC.
  • Demonstrate how all the components above work together.
  • Deploy the application into a Kubernetes cluster using Coherence Operator.

Although I will use Helidon, JavaFX, and React to implement the application, the focus will be very much on Coherence CE usage and APIs. Everything else is secondary, but you are more than welcome to explore the source code for the complete application, which is available on GitHub.

Fair warning: I am not a JavaFX or React expert, so some of the UI-related code may be suboptimal.

As you can see from the list above, there’s too much to fit into a single article. So, I will implement the REST API back end for the application here, and I’ll leave front-end implementations and operational aspects, such as deployment and monitoring, for future articles in this series.

Implementing the REST API

The first step is to create a Maven project with all the needed Helidon and Coherence CE dependencies. Do that by following the instructions in the Helidon MP Tutorial, and then add a few additional dependencies.

The resulting POM file should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">




    <!-- Helidon dependencies-->

    <!-- Coherence dependencies -->




In addition to the Helidon MicroProfile Bundle, I have added Eclipse Jersey support for JSON-B serialization and SSEs. The code also configures the Jandex plugin to index the application’s classes at build time to speed up Contexts and Dependency Injection (CDI) startup.

On the Coherence CE side, here are the three added modules:

  • Coherence CDI Server, which provides a CDI extension that starts the Coherence server within the application process, enables the injection of Coherence CE maps into Helidon services, and maps Coherence CE events to CDI events so they can be handled using standard CDI observers
  • Coherence MicroProfile Config, which will configure Coherence CE using the Helidon MP Config implementation
  • Coherence MicroProfile Metrics, which will publish Coherence CE metrics into the Helidon Metric Registry, making them accessible to monitoring tools such as Prometheus

I used a Maven property to specify not only the Coherence CE version but also the Maven groupId for Coherence CE dependencies. This is a recommended practice, because it allows you to easily switch between the open source version (Coherence CE) and the commercial versions (Oracle Coherence Enterprise Edition or Oracle Coherence Grid Edition), which have a different groupId. Everything else in your code, from artifact names to package and class names, can stay exactly the same.

In addition to creating a Maven project, it’s necessary to create a few configuration files. First, turn the application into a proper CDI bean archive by creating the META-INF/beans.xml file:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://xmlns.jcp.org/xml/ns/javaee"

It’s necessary to configure Java Logging to see log output from Helidon and Coherence CE. To configure it, add the logging.properties file to the src/main/resources directory:


# Global default logging level. Can be overriden by specific handlers and loggers

# Formatter configuration
java.util.logging.SimpleFormatter.format=%1$tY.%1$tm.%1$td %1$tH:%1$tM:%1$tS %4$s %3$s !thread!: %5$s%6$s%n

Implementing the domain model

To implement the REST API for the To Do List application, the application will need two things: a data model representing a single task in a to-do list and a JAX-RS resource that provides the necessary REST endpoints.

The first one is a trivial plain old Java object (POJO) with a handful of attributes that are very much self-explanatory:

public class Task
        implements Serializable
    private String id;
    private long createdAt;
    private String description;
    private Boolean completed;

     * Construct Task instance.
     * @param description  task description
    public Task(String description)
        this.id = UUID.randomUUID().toString().substring(0, 6);
        this.createdAt = System.currentTimeMillis();
        this.description = description;
        this.completed = false;

    // accessors omitted for brevity

Note that this POJO is serializable using Java serialization, which is what Coherence CE will use as a storage format, and using JSON-B, which will be used by the REST and gRPC APIs. Coherence CE supports pluggable serializers for both storage and the client/server communication. I could have used JSON for both. The application could also have used JSON as a transport format and Coherence Portable Object Format (POF) as a storage format.

The goal here is to demonstrate that you can use different serialization formats for client/server communication and storage, with Coherence CE automatically converting objects as necessary. It is a bit too early to talk about Coherence POF, though; that will come up later.

Implementing CRUD endpoints

With the data model in place, start implementing the REST API:

public class ToDoResource
    private NamedMap<String, Task> tasks;

    // TBD

As you can see in the code above, this JAX-RS resource will handle requests to the /api/tasks HTTP endpoint. It uses Coherence CE’s CDI support to inject a Coherence CE NamedMap to be a data store for the tasks.

Coherence CE’s NamedMap is an extension of java.util.Map, an interface many developers know. This is both good and bad. It is good because developers already know how to do many things with it. It is bad because developers may do things in a suboptimal way when a better alternative is readily available.

To dig deeper, unlike Java’s Map, Coherence CE’s NamedMap is a distributed data structure. Data in it is not stored within local buckets, as it is in a Java HashMap, for example. Instead, it is stored within partitions, which can be distributed across many JVMs, machines, or even data centers. This means that some of the operations you take for granted when you work with a local map are not nearly as efficient when you work with a NamedMap.

Take iteration as an example: You wouldn’t think twice about iterating over the entries of a HashMap. However, iterating over a NamedMap can be an extremely expensive operation, potentially moving gigabytes or even terabytes of data across the network, deserializing it, processing it, garbage collecting it, and so on.

Fortunately, most things work as you would expect, and the Coherence CE team has done a lot of work to make sure that’s the case. For example, the team has reimplemented many of the Map methods to use Coherence CE primitives, such as aggregators and entry processors, in a way that is suitable for a distributed Map implementation. Similarly, the developers have completely reimplemented the Stream API on top of Coherence CE aggregators, effectively allowing you to perform stream operations in parallel across many machines. Distributed lambdas have been reimplemented to allow them to be serialized using any of the supported serialization formats in a way that can be safely used in a distributed environment, with potentially different versions of those lambdas being present on different cluster members.

The point is that even though the Coherence CE developers have done all that optimization, you still need to remember that you are working with a distributed data structure and that most calls will end up on the network. Therefore, you should use Coherence CE features that allow you to minimize the impact of that.

Now, it’s time to implement the needed REST endpoints.

First, the code needs some basic create, read, update, delete (CRUD) functionality: the ability to add, update, and remove individual tasks and to retrieve a list of all tasks. I’ll start with the last of those items. The application will return a list of all tasks from the JAX-RS resource when it’s accessed via an HTTP GET request:

public Collection<Task> getTasks(@QueryParam("completed") Boolean completed)
    Filter<Task> filter = completed == null
                          ? Filters.always()
                          : Filters.equal(Task::isCompleted, completed);

    return tasks.values(filter, Comparator.comparingLong(Task::getCreatedAt));

This code shows an example of an optimization I mentioned earlier. Instead of fetching all the tasks, I use an overload of the Map.values() method, which executes the query in parallel across the cluster members and returns only the subset of the tasks with the specified completion status. It also sorts the results based on task creation time, so the results return to the clients in a consistent order.

If the client doesn’t specify the completion status to query on, return all tasks by passing an AlwaysFilter instance to the method.

A few of the methods are no different from what you’d have if you used a plain Map as a data store:

public void createTask(Task task)
    task = new Task(task.getDescription());
    tasks.put(task.getId(), task);

public void deleteTask(@PathParam("id") String id)

The REST endpoints allow clients to create and delete tasks using the Map.put and Map.remove methods. These are still remote calls, but there is really nothing you can do about that.

One thing to note is that the application is creating a new Task within the createTask method to control how new tasks are initialized in one place within the server implementation. This allows clients to send a minimal JSON representation of a new task containing only the description and to not worry about how the ID, creation time, and completion status are initialized.

Now for a slightly more interesting piece of functionality: bulk removal of completed tasks:

public void deleteCompletedTasks()
    tasks.invokeAll(Filters.equal(Task::isCompleted, true),

This topic deserves some additional discussion, because many developers haven’t seen an invokeAll method on a Map before.

NamedMap.invoke and NamedMap.invokeAll are the methods that support one of the most important distributed computing primitives you will ever use with Coherence CE: entry processors.

Understanding entry processors

Entry processors allow you to send functions into the cluster and process data in place, which completely eliminates the need to move the data over the network in most cases. In other words, the function moves to the data, not the other way around!

The code sent a ConditionalRemoveProcessor configured to remove all the entries that it executes against (by passing an AlwaysFilter as an argument), to execute against all the tasks in the NamedMap that have been completed. Another approach, by the way, would have been to target all entries by passing only an entry processor as an argument, a set of entries by specifying a set of keys to execute against, or a single entry by calling invoke instead of invokeAll.

To some extent, you can compare entry processors to stored procedures: Entry processors efficiently process large data sets in parallel without moving any data over the network (apart from any processing results, which you can optionally return). However, there are two important differences between them:

  • Entry processors are written in Java instead of SQL, because Java is to Coherence CE what SQL is to a database, and this makes them much easier to write and use (for Java developers, at least) than stored procedures could ever be.
  • Entry processors can be predefined via an existing Java class implementation that is available on both the client and the server. Or, they can be created dynamically via Java lambdas, in which case they are shipped from the client to the server, bytecode and all, and registered as versioned classes to support multiple versions of the same lambda across the cluster. The example above uses an existing class from Coherence CE itself, but I’ll show a lambda example shortly.

I cannot stress enough how important and powerful entry processors are. In theory, you could implement every other method in the NamedMap API using entry processors, and as a matter of fact, I do use them to implement many of the default Map methods that were added in Java 8.

Now for the next REST endpoint, which is the one that will update existing tasks:

public Task updateTask(@PathParam("id") String id, Task task)
    String  description = task.getDescription();
    Boolean completed   = task.isCompleted();

    return tasks.compute(id, (k, v) ->

            if (description != null)
            if (completed != null)
            return v;

This code may not look like an entry processor call on the surface, but if you dig a little deeper, you will see that NamedMap.compute is really just an entry processor in disguise.

default V compute(K key, BiFunction<? super K, ? super V, ? extends V> remappingFunction) 
    return invoke(key, CacheProcessors.compute(remappingFunction));

What’s interesting in the code above is that the implementation is no different from what it would be if you were using a local Map: Simply call Map.compute and update the description or the completion status based on the REST payload. What is different is where that code gets executed. In this particular case, the entry processor will ensure that it gets executed on the cluster member that owns the task with the specified ID.

By the way, entry processors are guaranteed to execute exactly once, even in the case of failures of primary or backup members, as long as the primary member and all its backups do not fail at the same time. This is an incredibly difficult guarantee to provide in a distributed system, and it’s a huge differentiator between Coherence CE and products from some of Oracle’s competitors, who may have a similar feature without the same guarantees Coherence CE provides.

With that, I’m mostly done with CRUD support within the REST API. The only remaining task is to implement support for SSEs, so clients can be notified when any of the existing tasks change or when tasks are added or deleted.

Implementing server-sent events

Coherence provides a rich, powerful event model: Applications can observe server-side events, which are raised on the member that owns the entry that triggered the event, and client-side events, which allow you to observe the events happening across the cluster.

What’s needed is to provide an SSE endpoint at /api/tasks/events that the REST API clients can use to register for a stream of change events to update their local state whenever the set of tasks in the Coherence CE NamedMap changes. All the clients should receive the same stream of events after the registration. The application leverages SSE broadcast support in JAX-RS to accomplish that:

private Sse sse;
private SseBroadcaster broadcaster;

void createBroadcaster()
    this.broadcaster = sse.newBroadcaster();

public void registerEventListener(@Context SseEventSink eventSink)

Once this code is added to the JAX-RS resource, the clients will be able to register with the SSE broadcaster. However, the application isn’t actually sending any events yet, although it can observe them.

Coherence CE events are represented using the Coherence CE MapEvent type, but the SSE broadcaster expects you to publish OutboundSseEvent instances with the event name and the relevant data as the payload. Fortunately, the solution is quite simple. Observe Coherence CE events, convert them to SSE events, and publish them:

void broadcastEvents(@Observes @MapName("tasks") MapEvent<String, Task> event)
    switch (event.getId())
        case MapEvent.ENTRY_INSERTED:
            broadcaster.broadcast(createEvent("insert", event.getNewValue()));
        case MapEvent.ENTRY_UPDATED:
            broadcaster.broadcast(createEvent("update", event.getNewValue()));
        case MapEvent.ENTRY_DELETED:
            broadcaster.broadcast(createEvent("delete", event.getOldValue()));

private OutboundSseEvent createEvent(String name, Task task)
    return sse.newEventBuilder()
                .data(Task.class, task)

I am using a client-side CDI observer to listen for all map events on the tasks map. Each of those events carries a payload containing the event type, the entry key, and the old and the new value of the entry, if applicable.

Only the update events will have both the old and the new value. The insert events will contain only the new value, and the delete events will have only the old value. Convert MapEvent into an SSE event by passing the appropriate event name and the payload to the createEvent method and broadcast it to all connected clients.

That concludes the REST API implementation.

I have implemented all the needed CRUD endpoints and discussed various Coherence CE features along the way. I have also implemented SSE support by converting Coherence CE events to SSE events and broadcasting them to registered clients.

But will it work?

Running the server

The easiest way to run the server is to simply run it within your IDE of choice, which in my case is IntelliJ IDEA (see Figure 2):

Server run configuration in IntelliJ IDEA

Figure 2. Server run configuration in IntelliJ IDEA

Once you have the run configuration defined, click the Run button and Helidon will bootstrap both its own web server, which will serve the REST API, and a Coherence CE cluster member that will be used to store the data.

Wait…What!? That is correct: Coherence CE is not a server that needs to be started, unlike most data stores you may already be familiar with from relational databases, such as Oracle Database and MySQL, or NoSQL key-value data stores such as Mongo and Redis. It is a Java library, which can be easily embedded into any Java application! (It can even be embedded into Node.js applications using GraalVM.)

That’s not to say that there is nothing that needs to be started. Coherence CE (the library) provides a set of services that need to be started in order for it to form the cluster and manage data, for example, the Cluster service and a PartitionedService. However, because these services are essentially just Java classes that implement a com.tangosol.util.Service interface, they can be easily started programmatically within the application.

That’s exactly what happens here.

The Helidon io.helidon.microprofile.cdi.Main class will simply configure logging and bootstrap the CDI container. The CDI container will then look for all the available CDI extensions, and it will discover both the Helidon Web Server extension and the Coherence CE extension, among others. Both of these will observe an @Initialized event that will be fired by the CDI container once it’s ready for use and will then start the Helidon Web Server and the Coherence CE server, respectively.

This architecture provides two significant benefits:

  • It allows you to reduce complexity by effectively combining an application server and a data store into a single stateful application server. This makes the architecture much simpler, because there are fewer elements to provision, scale, monitor, and manage. This tends to work quite well for simple applications and services. That is what I mean when I talk about the combination of Helidon and Coherence CE as a stateful microservices platform.
  • The architecture makes it easy to debug the code that executes within Coherence CE, such as entry processors, especially now that the actual, fully documented source code for Coherence CE is available on GitHub and on Maven Central.

Now, that doesn’t mean you always have to run the application that way. You can certainly choose to split the application server and the data store using Coherence CE roles and then manage them separately, but this is truly a deployment-time decision, not a development-time one. You can develop the application, run it within the IDE as a single process for testing and debugging, and even package it into a single Docker image. Then later you can choose to run separate app server and storage tiers by changing how you deploy the application to Kubernetes.

You can also run different microservices in a single Coherence CE cluster, or in multiple clusters, or even as a quasi-monolith by running all the services in every cluster member. The possibilities are endless, and the best option depends on your particular use case.

However, it is hard to overstate how much simpler the typical developer workflow is by simply embedding the data store into the app. Try it and you’ll see.

Go back to actually running the To Do List server. If you clicked on the Run button earlier, you should see log output similar to the following towards the end:

2020.07.30 05:55:53 INFO io.helidon.microprofile.server.ServerCdiExtension Thread[main,5,main]: Server started on http://localhost:7001 (and all other host addresses) in 12215 milliseconds (since JVM startup).

That output means the Helidon Web Server was successfully started and can be accessed on port 7001. You should also see the following a bit higher up in the log:

2020.07.30 05:55:53 CONFIG org.glassfish.jersey.server.ApplicationHandler Thread[main,5,main]: Jersey application initialized.
Root Resource Classes:

That output means the JAX-RS resource was discovered and deployed by Helidon.

Finally, you should be able to scroll a bit higher up and find the following:

2020.07.30 05:55:52 INFO coherence Thread[Logger@9258732 20.06,3,main]: (thread=DefaultCacheServer, member=1, up=10.634): 
  // omitted for brevity
Started DefaultCacheServer...

That output means the Coherence CE server was started successfully and the application can actually store data in Coherence CE.

Much more information is logged by both Helidon and Coherence CE, so feel free to explore. However, these three sections in the log are critical, and if they are present, the REST API is ready to use.

Testing the REST API

I could’ve implemented automated tests for the REST API using JUnit and REST Assured, but for the time being, I use the curl tool to see if everything works as expected.

There’s no data in the application yet, so I’ll create some tasks using the POST endpoint:

$ curl -i -X POST -H "Content-Type: application/json" \
       -d '{"description": "Learn Coherence"}' http://localhost:7001/api/tasks
HTTP/1.1 204 No Content
$ curl -i -X POST -H "Content-Type: application/json" \
       -d '{"description": "Write an article"}' http://localhost:7001/api/tasks
HTTP/1.1 204 No Content

Based on the HTTP response codes, that seems to be working. To check, run this command:

$ curl -i -X GET -H "Accept: application/json" http://localhost:7001/api/tasks
HTTP/1.1 200 OK
Content-Type: application/json

[{"completed":false,"createdAt":1596105616507,"description":"Learn Coherence","id":"9c6d9a"}
,{"completed":false,"createdAt":1596105656378,"description":"Write an article","id":"a3f764"}]

That output looks good. Can this application complete a task?

$ curl -i -X PUT -H "Content-Type: application/json" \
       -d '{"completed": true}' http://localhost:7001/api/tasks/a3f764
HTTP/1.1 200 OK
Content-Type: application/json

{"completed":true,"createdAt":1596105656378,"description":"Write an article","id":"a3f764"}

Awesome. The output shows that worked as well.


The article created a To Do List application using Coherence CE and Helidon, and it appears to be working correctly. I’ll leave the deletion of completed tasks and the deletion of tasks by ID as an exercise for readers, but you get the idea: You can store, update, delete, and retrieve tasks from Coherence CE using the REST API developed earlier.

Although everything is working fine as long as you have the server up and running, you’ll notice that if you shut down the server you will lose all the data.

This is expected, because you haven’t configured Coherence CE to persist data to disk yet. So at the moment, data is stored only in memory and it’s lost if you restart the cluster, which currently has a single member. (I know: It’s not much of a “cluster” at this point, is it?)

One way to fix that is by starting more members, which will automatically enable high-availability mode and create backups of the tasks. Unfortunately, I can’t do that at the moment. Any additional members would fail to start because of TCP bind exceptions, because Helidon will attempt to start the web server on the same port (7001). That problem will go away in the final article of this series, which deploys the application to Kubernetes.

You can also prevent data loss on restart by enabling disk persistence, which can be accomplished by passing the -Dcoherence.distributed.peristence.mode=active system property on the command line, but I wouldn’t worry about it for now. It is faster to start the server with persistence disabled, and it’s usually simpler to start testing with a clean slate.

It is now time to build a decent looking front end for the application, so nobody will have to use curl to manage tasks ever again. However, I’ll leave that for the next article in this series.

Aleks Seović

Aleks Seović (@aseovic) is an architect at Oracle, where he works on Oracle Coherence, a leading in-memory data grid product, and contributes to the Helidon microservices framework. Most recently, Seović led the design and implementation of the Helidon gRPC framework, as well as CDI and Eclipse MicroProfile support in Coherence. He currently leads the implementation of Coherence native clients, GraphQL support, and Spring integration. Prior to joining Oracle in 2016, Seović led a boutique consultancy practice, where he worked with customers around the world to help them implement mission-critical applications on top of Coherence. Seović is the author of Oracle Coherence 3.5 (Packt Publishing, 2010) and frequently speaks about and evangelizes Coherence at industry conferences, Java and .NET user group events, and Coherence SIGs.

Share this Page