X

An Oracle blog about Oracle Coherence

Recent Posts

Coherence Community

Updated Coherence Demo now runs on Kubernetes

Introduction We are happy to announce that a new version of the Coherence Demo has just been released which can run against a Kuberenetes environment. This demo utilizes a new open source project the Coherence Operator which helps you deploy and manage Coherence within a Kubernetes environment. The Coherence Operator is available on GitHub at https://github.com/oracle/coherence-operator while the Coherence Demo is available at https://github.com/coherence-community/coherence-demo. While updating the Coherence Demo, we've ensured that it still runs stand-alone, or on a single machine as it did before. Pre-Requisites The full set of pre-requisites are available in the Coherence Demonstration README, but as a quick guide I've included main requirements below: Kubernetes 1.11.5+, 1.12.3+, 1.13.0+ Docker 18.03.1-ce  Helm 2.12.3  Coherence 12.2.1.3.2+ or 12.2.1.4.0+ (for Federation support) If you wish to run the Coherence Demo on your Mac, you can with Docker Desktop version 2.0.0.3 and above. Building and Running the Coherence Demo The instructions below are summarised, with the assumption of all Coherence Demo pre-requisites met and using Docker Desktop on a MacBook Pro. Note: You can follow the full instructions which includes pre-requisites and instructions how to enable Federation, in the README.md for the coherence-demo project. Clone the repository using the following command: $ git clone https://github.com/coherence-community/coherence-demo.git Once you have your build environment set and Docker running, then build the Maven project using: $ mvn clean install -P docker The above command, will create a docker image which contains the cache configuration and java classes to run the demo application. Install the Coherence Operator Chart Issue the following to install the Coherence Operator chart: helm install \ --namespace coherence-demo-ns \ --set imagePullSecrets=coherence-demo-secret \ --name coherence-operator \ --set "targetNamespaces={coherence-demo-ns}" \ coherence/coherence-operator Install the Coherence Chart $ helm install \ --namespace coherence-demo-ns \ --name coherence-demo \ --set clusterSize=1 \ --set cluster=primary-cluster \ --set imagePullSecrets=coherence-demo-secret \ --set store.cacheConfig=cache-config.xml \ --set store.pof.config=pof-config.xml \ --set store.javaOpts="-Dprimary.cluster=primary-cluster" \ --set userArtifacts.image=coherence-demo-sidecar:3.0.0-SNAPSHOT \ coherence/coherence Once installed, ensure that the pods are all running by using the following. Once the coherence-demo-0 pod has READY of 1/1, you can continue. $ kubectl get pods --namespace coherence-demo-ns NAME READY STATUS RESTARTS AGE coherence-demo-0 1/1 Running 0 54s coherence-operator-dc7867dbc-n 1/1 Running 0 54m Port-forward the application HTTP port (8080) so that we can access the application via localhost (127.0.0.1). $ kubectl port-forward --namespace coherence-demo-ns coherence-demo-0 8080:8080 Open the following URL to access the application home page - http://127.0.0.1:8080/application/index.html. When running the stand-alone version of the Coherence Demo (See here), you could change the cluster size by adding or removing servers from the UI. When running under Kubernetes, you must the the kubectl command to scale the cluster. For example, to scale the Coherence Demo cluster to 3 servers, you would use the following: $ kubectl scale statefulsets coherence-demo --namespace coherence-demo-ns --replicas=2 You can use the following, (or view the application page), to see when the cluster has been scaled. $ kubectl get statefulsets coherence-demo --namespace coherence-demo-ns NAME DESIRED CURRENT AGE coherence-demo 2 2 16m The application should show 2 members as below. To shutdown the Coherence Demo and Coherence Operator you use following helm command: $ helm delete coherence-demo coherence-operator --purge See the following links for more information on the Coherence Operator and Coherence Demo: Coherence Operator on GitHub - https://github.com/oracle/coherence-operator Coherence Operator Documentation - https://oracle.github.io/coherence-operator Coherence Demo on GithHub - https://github.com/coherence-community/coherence-demo Coherence Documentation - https://docs.oracle.com/middleware/1213/coherence/index.html  

Introduction We are happy to announce that a new version of the Coherence Demo has just been released which can run against a Kuberenetes environment. This demo utilizes a new open source project the Co...

Coherence Operator 1.0 Released! Operate Coherence Clusters with Kubernetes.

Oracle Coherence Product Development is pleased to announce the release of Coherence Operator 1.0 into general availability. Coherence Operator is a Kubernetes Operator for Coherence clusters: a Coherence-specific controller extending the Kubernetes API to create, configure, and manage Coherence deployments. Coherence's ability to provide a shared cache, and its inherent dynamic scalability, align naturally with Kubernetes-based deployments of microservices architectures. With Coherence Operator, Coherence users can leverage de-facto industry standards Docker for containerization, and Kubernetes for orchestration of containers, both on-premise and in cloud environments equally.  Specifically, Coherence users can: Use popular de-facto standard tools to monitor and analyze logs and metrics from Coherence clusters.  The Elastic Stack is supported now; Prometheus and Grafana will be supported in the upcoming Coherence 12.2.1.4 release. Flexibly override and customize Coherence configuration Scale the Coherence deployment using Kubernetes verbs or updates Use Coherence*Extend and REST to access Coherence clusters from a variety of clients Use Kubernetes Zones to ensure data stored in Coherence is resilient to Zone loss Start clusters based on declarative startup parameters and desired states Use Kubernetes persistent volumes when using Coherence’s disk-based storage features Deploy custom application code to server-side Coherence JVMs “With this investment in Kubernetes orchestration of Coherence clusters running in Docker containers, we are embracing and facilitating a growing trend we see in our customer base,” said Alex Gleyzer, Vice President, Software Development, Oracle Coherence.  “Leveraging an industry-standard framework for managing distributed systems allows our customers to operate Coherence deployments more efficiently and reliably than ever before.” “Kubernetes is key to Oracle’s Enterprise Java strategy, enabling customers to move applications from on-premise to in-cloud deployments,” said Will Lyons, Senior Director, Product Management, Enterprise Java.  “Kubernetes Operators for WebLogic Server, and now Coherence, facilitate moving Enterprise Java applications onto Oracle Cloud Infrastructure, or other public and private clouds, using cloud-neutral Kubernetes infrastructure.” Coherence Operator is open-source software, hosted on GitHub, and maintained by Oracle Coherence Product Development. Docker images containing Coherence Operator releases are available on Docker Hub. Full documentation on Coherence Operator, including a Quick Start Guide, a User Guide, a rich set of samples, and more, is hosted on GitHub Pages.  In addition Oracle Coherence Product Development has created a public Slack channel for communicating with users of Coherence Operator.  Instructions for joining the Slack channel are included in the Coherence Operator documentation. The Coherence Demo is enhanced with instructions for running with Coherence Operator.  And the Oracle Coherence YouTube channel has new videos overviewing Coherence Operator, and providing detail on management with Kibana and Grafana, and using persistence and federated caching in Kubernetes. Oracle Coherence Product Development invites you to use Coherence Operator 1.0 and provide feedback for improving future releases.  We are standing by to support you.

Oracle Coherence Product Development is pleased to announce the release of Coherence Operator 1.0 into general availability. Coherence Operator is a Kubernetes Operator for Coherence clusters: a...

Coherence

Review: Oracle Coherence at OpenWorld and JavaOne

Three monthsprep.  Then its here. Present, eat, drink, sleep when possible.   Welcome to the OpenWorld/JavaOne cycle. Coherence is thebasis for several Oracle Cloud Services, and, so, we were a busy team at OracleOpenWorld and Java One. This year had noticeableemphasis on cloud use-cases, microservices, cloud-native, and cloudservices. Whether it was Union Pacific talkingabout how they use Coherence for state management in their on-premisesmicroservices architecture, how Coherence fits onto the Oracle Cloud AppDev story,or new programming techniques that cater to cloud-native programming,customer’s move to the cloud and dominated the agenda. Between OpenWorld andJavaOne, we presented eight Coherence-related sessions. Here is a review of what we presented, withlinks to downloadable presentations. Our roadmap and strategy talk, Oracle Coherence Innovationsfor Modern On-Premises and Cloud Applications, reviews Coherence 12c features such as Persistence and Federated Caching, as well as updates on features more recently released via our community process such as a CoherenceRx, a reactive programing interface, and Docker support.  This presentation combined with our cloud caching and data grid services roadmap talk details what we have been working on in the lab. Union Pacific took us on their trainride from Mainframe to Thousands of Microservices and shared how they Managed Microservice State with Coherence.  Together, these two presentations discuss the evolution of organizational thinking and structure as well as technical challenges of building microservice applications.   In-memory data grids serve many roles in cloud native architectures, from caching to state management.  We examined these benefits of IMDG in the cloud by first looking at what cloud native architectures are, and then showing how data grids like Coherence have long served roles in state management and the other application architecture challenges posed by the cloud.  Some myths of "stateless applications" are blown up in an excellent JavaOne talk The Illusion of Statelessness.  You can also watch this talk on YouTube. And finally some goodies for hardcore developers.  Cloud native, 12-factor apps make a lot more use of async programming than we have seen in the past, and Java is keeping up.  At JavaOne, we presented a BOF on The State of Asynchronous Programming in Java: Completable or Reactive Future?  We also presented a talk on how we solved the problem of extending Java 8 Lambdas and Streams to Distributed Systems in such a way that you can safely use them in production to improve programming productivity and simplify operations.  Beware cheap imitations! ·

Three months prep.  Then its here. Present, eat, drink, sleep when possible.   Welcome to the OpenWorld/JavaOne cycle. Coherence is thebasis for several Oracle Cloud Services, and, so, we were a...

Coherence

Announcing Coherence 12.2.1.1.0

Oracle Coherence 12.2.1.1.0 is here!  That's a lot of numbers.  Or three, depending on how you count.  What it really means is that this release is a patch-set and not a large new release.  The release is packed full of goodies for users of the new Federated Caching and Persistence features released in October 2015 as part of Coherence 12.2.1, and also contains many improvements for users of any prior Coherence release. Here is a summary of the changes.  For a full description of the changes in 12.2.1.1.0, check our release notes.   Federated Caching Improvements  An address port is no longer required when configuringFederation participants. Cache entries that are loaded into a cache using aCacheStore implementation can now be federated across clusters. The DestinationMBean now includes managementinformation for replicateAll operations. HTTP Session caches can now be federated.  Federated caching can be disabled for specific caches.  Coherence Session API The Coherence Session API provides applications with a new and recommended interface to get a reference to a NamedCache instance.  Proxy Tier Management Information New Proxy Tier Management Information includes the number of connections across each proxy server and the totalmessages that were sent and received. Management attributes are included on theConnectionManagerMBean and are viewable on the Proxy HTTP Report and theHTTP Servers tab in the Coherence-JVisualVM Plug-in.  Two Member Partition Assignment The default partition assignment strategy now uses anactive-passive distribution algorithm for clusters with only twostorage-enabled members.   Simplified Configuration of Persistence and Federation with Managed Coherence Servers Persistence and federation can now be configured in theWebLogic Server Administration Console or using WebLogic Server MBeans. Formore information, see ConfiguringCache Persistence and Configuring Cache Federation. Zero Downtime Support Managed Coherence Servers can now participate in Zero Downtime Patching.  IBM WebSphere Liberty Support Coherence*Web supports IBM WebSphere Liberty 8.5 and higher.Support for Coherence*Web with other IBM WebSphere versions has been removed. See SupportedWeb Containers for a full list of supported containers for Coherence*Web. Dynamic Active Persistence Quorum Policy The dynamic recovery quorum policy is used with active persistence and automatically configures the persistence recovery quorum based on a predefined algorithm. 

Oracle Coherence 12.2.1.1.0 is here!  That's a lot of numbers.  Or three, depending on how you count.  What it really means is that this release is a patch-set and not a large new release.  The...

Coherence

CoherenceRx: Reactive Extensions for Oracle Coherence

We are pleased to announce the release of a new Oracle Coherence Community open source project.  The CoherenceRx project provides a reactive API for the Oracle Coherence in-memory data grid, based on the popular RxJava library. It is implemented as a thin wrapper around the Oracle Coherence Asynchronous API, which implies that it requires Coherence 12.2.1 or a newer release. Why CoherenceRx? Reactive Programming is somewhat of an all-or-nothing proposition, or as Andre Staltz pointed out in his excellent tutorial: When you are writing a reactive application and need to access a data source that doesn't provide a reactive API, life can get complicated. In order to simplify our users' lives we decided to implement CoherenceRx and release it as an open source add-on for Coherence. Using CoherenceRx The easiest way to include CoherenceRx into your own project is to add it as a Maven dependency (along with Coherence itself and RxJava): <dependency> <groupId>com.oracle.coherence</groupId> <artifactId>coherence</artifactId> <version>${coherence.version}</version></dependency><dependency> <groupId>com.oracle.coherence</groupId> <artifactId>coherence-rx</artifactId> <version>${coherence-rx.version}</version></dependency><dependency> <groupId>io.reactivex</groupId> <artifactId>rxjava</artifactId> <version>${rxjava.version}</version></dependency> and configure versions within Maven properties section: <coherence.version>12.2.1-0-0</coherence.version><coherence-rx.version>1.0.0</coherence-rx.version><rxjava.version>1.1.0</rxjava.version> Once you have the necessary dependencies properly configured, you can use the static RxNamedCache.rx method to create an instance of RxNamedCache: NamedCache<Long, Product> cache = CacheFactory.getTypedCache("trades", withTypes(Long.class, Product.class));RxNamedCache<Long, Product> rxCache = RxNamedCache.rx(cache); Of course, you can also use static import for the RxNamedCache.rx method, which would make the code even simpler. The RxNamedCache interface will be familiar to anyone who has used Coherence NamedCache API before, with one major difference: all the methods return an Observable. For example, RxNamedCache.get will return an Observable<V> which will eventually emit the value of the cache entry for the given key and complete: rxCache.get(5L).subscribe(product -> System.out.println("Got: " + product)); Another important difference is that the bulk read operations, such as getAll, keySet, entrySet and values do not return a single container value like their NamedCache counterparts, but an Observable stream of individual values: rxCache.values().subscribe(product -> System.out.println("Got: " + product)); This is both more efficient, as it doesn't realize full result set on the client, and simpler, as it allows you to process each individual value as it is emitted by the underlying Observable. For example, if you wanted to process batches of 10 products at a time, you could trivially accomplish that using bufferoperation: rxCache.values() .buffer(10) .subscribe(productList -> System.out.println("Got: " + productList)); Observing Event Streams Oracle Coherence provides rich event notification functionality, so it only made sense to provide an adapter that allows you to use RxJava to process stream of event notifications. CoherenceRx introduces ObservableMapListener, which extends RxJava Observable and implements CoherenceMapListener interface. The ObservableMapListener simply propagates each received event to all of its subscribers: ObservableMapListener<Long, Product> listener = ObservableMapListener.create();listener.subscribe(System.out::println);cache.addMapListener(listener); The above is not very interesting, and could be easily achieved using standard SimpleMapListener. But it becomes a lot more interesting when you start applying various RxJava operators to transform, filter and even combine event streams: ObservableMapListener<Long, Trade> listener = ObservableMapListener.create();listener.filter(evt -> evt.getId() == MapEvent.ENTRY_INSERTED) .map(MapEvent::getNewValue) .buffer(10, TimeUnit.SECONDS) .subscribe(trades -> System.out.println("Trades placed in the last 10 seconds: " + trades));cache.addMapListener(listener); It is important to note that unlike Observables returned by the RxNamedCache methods, which are 'cold', the ObservableMapListener is a 'hot' Observable and will start receiving and processing the events as soon as it is registered with the cache using NamedCache.addMapListener method. Because of that, it is important that you add Subscribers to it before calling NamedCache.addMapListener, or you could miss some events. Feedback CoherenceRx is the latest example of how Oracle Coherence continues to improve the developer experience when working with In-Memory Data Grid platforms.  In the past several months, we have provided groundbreaking support for Distributed Java 8 Lambdas and Streams, a new Async API based on CompletableFutures, and, now, support for Reactive Programming.  We welcome feedback on the CoherenceRx project in the comments section below, or, as you use it, you can file issues on the GitHub project page.

We are pleased to announce the release of a new Oracle Coherence Community open source project.  The CoherenceRx project provides a reactive API for the Oracle Coherence in-memory data grid, based on...

Coherence

Coherence SIGs in Europe in December

Coherence and WebLogic Server team members will be in Europe the week of December 7 talking about the new Coherence release  as well as new WebLogic Server capabilities that include integration with Coherence such as Multitenancy and Continuous Availability.   For both Coherence and WebLogic, these are landmark releases delivering unique capabilities for both developers (distributed Java 8!) and ops (Multitenancy, Persistence, Federated Caching, Continuous Availability).  We look forward to sharing these new features and getting feedback from our community. Munich Our week starts in Munich with a Cloud Application Foundation SIG on Tuesday December 8. Full information and registration is available here.  Here is the planned agenda: 9:30 Coffee and Registration 9:45 Introduction to WebLogic Server and Coherence 12cR2 Multitenancy and Continuous Availability 10:30 Fueling Applications with Coherence 12cR2 11:15 WebLogic Server Customer/Partner Presentation 12:00 Lunch (and more coffee!) 12:45 Innovating with Java: Benefiting from Distributed Lambdas and Streams in Coherence 13:45 Q&A Madrid  On Thursday December 10 we will be in Madrid.  Email Marta.Salso to register or for details of this event.  The following is a preliminary agenda: 9:00  Welcome and coffee 9:30 Coherence 12.2.1  10:30 Coherence customer use case  11:00 Break & networking 11:30 Java 8 with Coherence Deep Dive (Distributed Lambdas and Streams) 12:30 Native JSON/JavaScript with Coherence & JSON/JavaScript London  The London Coherence SIG will be held at our London City Office on Friday December 11.  Full information and registration information is available here.  Here is the planned agenda: 10:00 Registration and Coffee 10:30 Introduction 10:45 Coherence 12.2.1 Update 12:00 Distributed Lambdas and Streams with Coherence Deep Dive 1:00 Lunch and Networking 1:45 Oracle WebLogic Server Multitenancy is Here! 2:30 Coherence Cache Persistence 3:15 Coffee and Networking 3:45 Using Coherence with JavaScript and JSON 

Coherence and WebLogic Server team members will be in Europe the week of December 7 talking about the new Coherence release  as well as new WebLogic Server capabilities that include integration with...

Coherence

Coherence Forums and Support

Do you know that you have ways of getting Coherence supportbeyond the standard Service Request process? We currently have multiple online forums where you can submit questionsabout Coherence. In an effort toencourage more open communication, the Coherence team is consolidating theseforums. The Coherence My Oracle Support Community (MOSC) has been available to customers with a paid Oracle support contract, and is lightly trafficked. This site will be retired. The Oracle Technology Network (OTN) support forums on https://community.oracle.com/community/fusion_middleware/coherence allow anyone to post and answer questions about Coherence. This is a more heavily used community forum, and we are encouraging increased usage of this resource. We hope that this consolidation will provide a more activeand open discussion around use cases, design questions, and issues. The Coherence MOS Community also provided convenient linksto other resources, and we would like to ensure that you know how to accessthese resources once the site is retired. As an Oracle customer with a current Support Contract hasaccess to MyOracleSupport , have access to Support Notes, Knowledge Documents, and the ability to log a Service Request. You can access all this through MOS: 1. Logonto MyOracleSupport (https://support.oracle.com) 2. Toaccess Knowledge Documents enter the product name and some search terms: 3.  Thisis the screen you see after clicking on Search: 4.  Clickon the title of the Knowledge Document you would like to view 5. Forthe most timely and official resolution of issues, please log a Service Requestwith from the Dashboard (Bottom right) or the Service Request tab. Thisrequires more detailed information about the product, version etc. We always encourage you to upload adescription of the use case, deployment information, configuration files, andlog files. For some Service Requests, wemay ask you for stack traces, memory dumps, GC logs or other information. The faster you can provide us with thisinformation, the faster we will be able to resolve any issue you may be having. 6.  TheNews and Getting started sections on the left are very useful for generalinformation about Oracle support. 7.  Accessto CoherenceInformation and Documentation. 8.  Asmentioned above, you can access the CoherenceOTN Community to ask questions to awide audience including partners and customers. 9.  Wehighly recommend that you look at the following knowledge documents: Oracle Support Accreditation - Series Index (Doc ID 1583898.1) Oracle Support Accreditation Series for Level 1 My Oracle Support (Doc ID 1579751.1) Oracle Support Accreditation Series - Oracle Fusion Middleware (Doc ID1584011.1)

Do you know that you have ways of getting Coherence support beyond the standard Service Request process? We currently have multiple online forums where you can submit questionsabout Coherence. In an...

Coherence

Oracle Coherence 12cR2 Offers Major New Innovations

We are very happy to announce major new innovations with the release of OracleCoherence 12.2.1 This release is one of thelargest we have done and delivers several major features aimed at bothdevelopers and operations that improve developer productivity, supportinfrastructure consolidation, provide better security, and simplifymulti-datacenter solutions. We are alsoannouncing a new licensing option for low-cost deployment of small-to-mediumsized deployments. New Features Coherence 12.2.1 is packed with major new features andcapabilities: The new Persistencefeature enables Coherence to be a system-of-record and improves time-to-recovery from planned or unplanned, partial or total,cluster outage. Persistence can be configuredto work in an on-line mode persisting changes as they are made to the cluster,or to take a snapshot of a data grid. Local disk configuration maximizes scalability, while network-attachedstorage maximizes availability.   Bothdata and metadata (indexes, listeners, expiry, locks, etc.) are stored, andrecovery is completely automatic. Federated Cachingprovides support for Active-Active, Active-Passive, Hub and Spoke, and custommulti-datacenter topologies. Implemented at the lowest level of Coherence,Federated Caching takes advantage of Message Bus, Elastic Data, and our newPersistence feature to improve scalability, durability, latency, andconfigurability of data replication over prior solutions.   Coherence Multitenancyallows WebLogic Server Multitenant applications to share a cluster across aset of tenants and applications, consolidating infrastructurerequirements. Multitenancy providestenant-specific caches as well as shared caches for hosting reference data.  It is fully integrated with Oracle TrafficDirector, WebLogic Server, and the Oracle Database to provide an end-to-end multitenantsolution. One of our most exciting new features is the integration wehave done with Java 8 to provide Distributed Lambdas and Streams. We now provide the ability to execute Lambdasand Streams remotely, using standard Java 8, distributed across clustermembers, without the need to deploy code on cluster servers. This provides the ability to do distributed,parallelized stream-based functional programming.  By reducing the need to deploy classes toservers, we have also greatly simplified application upgrade. New Security Features include the ability to do fine-grained Authorization and Auditing of all cache access and updates.   We have also improved our SSL support. For developers, we have also added support for Asynchronous Cache operations via Java 8 CompletableFutures for quicker,parallel execution of multiple tasks as well as a Streaming Async API to improve performance and reduce memorypressure on bulk reads. We also offer improved Generics Support. Customers will notice increasedperformance from near cache optimizations and automatic, dynamic threadpool tuning.  We have improvedcluster and port management, and switched data communications to run overTCP via the TCP Message Bus. We alsoprovide Dynamic Coherence Management Node for Management HA. We also have a number of other goodies, including supportfor Sliding Expiry, a new cache truncate operation, REST Server-Sent Events, and JSON Passthrough that improve the developer experience and offer new functionality. New Packaging andLicensing Options We also offer new (and sometimes free!) options for deploying Coherence in production. Coherence Standard Edition One replaces StandardEdition.   SE One is a low-priced ($800 /socket) dual-server Active/Standby solution that provides all Coherence APIsfrom Java, REST, JCache, and Memcached clients. It is perfect for smaller deployments where you data fits into a singlenode, e.g. anywhere up to around 20GB or larger depending on your tolerance forlarger GC pauses and need for scale-out. SE One also includes the new Persistence features. The good news around SE One does not end there.  SE One is included in Java SE Advanced (withthe limitation that it is a single-server deployment with no standby). Since WebLogic Server Enterprise Edition includes Java SEAdvanced, WebLogic customers can now deploy Coherence for free inconfigurations not needing scale-out or advanced features of Enterprise and GridEdition. Customers needing the scale-out or other features ofCoherence Enterprise Edition or Grid Edition can upgrade without code changesince SE One provides the full Coherence API. Real-Time Extend Data Clients are now included for free inevery Coherence edition to provide deployment flexibility.  No longer do you need to decide whether youwill use near cache or whether you will use an Extend proxy before you choosethe appropriate Coherence edition. Persistence is part of every Coherence edition.  WebLogic Server Multitenant and WebLogic Server ContinuousAvailability  both provide the right toupgrade from Enterprise Edition to Grid Edition. If you own Coherence Enterprise Edition orWebLogic Suite and by WLS MT or CA, you will be able to deploy Coherence GE totake advantage of multitenant and continuous availability features of Coherencein your WLS applications. Finally, we have improved our Free Developer’s License onOTN to allow developers to use Coherence for free in development even afterCoherence has been deployed in production.  I hope this has whet your appetite for Coherence12.2.1. As you can tell, I am quiteexcited about it. Please check Coherence12.2.1 Documentation for more information on the features above and watchas deep-dive videos from our engineering team are added to the YouTube OracleCoherenceChannel over the next few days. 

We are very happy to announce major new innovations with the release of Oracle Coherence 12.2.1 This release is one of the largest we have done and delivers several major features aimed at bothdevelope...

Coherence

Oracle Coherence recognized as a leader in new Forrester Wave report

Forrester Research has published The Forrester Wave™: InMemory Data Grids, Q3 2015. The report,which is subtitled “There Is No Better Way To Achieve Blazing Fast PerformanceAt Scale” analyzes features, market presence, and roadmap of eleven vendors inthe In-Memory Data Grid space.Oracle is cited as a leader in the report.  In the report, OracleCoherence is top ranked among the 11 vendors forCurrent Offering (Oracle Coherence 12.1.3)Solution RoadmapMarket Presence Mike Gualteri, author of the report, writes about Oracle’s performance:“Oracle innovates to stay on top of the category it invented. 2007seems like a generation ago in technology years, but that is when Oracleacquired Tangosol arguably the company that created the market for in-memorydata grids that were more than just data caches. The IMDG that started it allcontinues to set the standard for all other IMDG vendors.” Coherence scores 5 out 5 possible points in the Security andData Features categories. Coherence alsoscores a 5 for Ability to Execute, Implementation Support, Customer Base, andPartnerships.Gualteri also states that IMDG provides “performance atscale.” “It’s the perfect storm — increasingapp functionality, more concurrent users, and legacy back-end componentry.In-memory data and compute grids are perfectly designed to help [ApplicationDevelopment and Delivery] professionals get through this storm with colorsflying proudly.”The fullForrester Wave report (userid/password required) details the role of IMDGin scalable architectures and when to think about it in place of NoSQLsolutions. The report explains use casesin caching, processing real-time data, and IMDG’s role in modern apps byserving as primary store-of-record.In-memory data grids (IMDG) like Oracle Coherence provide clustered,scalable, low-latency access to application data. IMDGsprovide query support, MapReduce aggregation, and in-place distributedprocessing. Oracle Coherence enables applicationscalability, data source offload, and real-time, fast data applications. Oracle Coherence pioneered IMDG in 2001, andhas been a market leader ever since.

Forrester Research has published The Forrester Wave™: In Memory Data Grids, Q3 2015. The report, which is subtitled “There Is No Better Way To Achieve Blazing Fast PerformanceAt Scale” analyzes...

Coherence

Coherence is in the Cloud as Part of Java Cloud Service

This entry is reposted from the new cloud-focused blog at Oracle, "The Cloud Front."Today we are excited to announce a major product innovation - Oracle Coherence is now available as a feature of the Java Cloud Service (JCS). Effective immediately, users of JCS can use our cloud tooling to optionally set up Coherence within JCS. Flexibility is a key part of the solution. You can either select an initial capacity for the Coherence data tier: Or use the "Advanced" option to have full control over the number of virtual machines you want for Coherence, the number of managed Coherence servers per virtual machine, and the heap size of each managed server:  The WebLogic Management Framework is used under the covers allowing you, the user to develop, deploy, manage and monitor your applications via Managed Coherence Servers. Because the same cloud tooling creates both WebLogic and Coherence, the experience of using the two together in our cloud is a fully integrated and seamless service. As is standard with JCS, you can patch Coherence as you please in an automated rolling fashion. Backups and restores are fully coordinated with the rest of the stack. Common reasons for using Coherence with JCS: Offload and protect shared cloud services and databases Deliver real-time data to cloud applications Provide transparency and high availability in the cloud's data grid tier When running a WebLogic 12.1.3+ environment in the Java Cloud Service you have access to all the Coherence cloud tooling. For Java Cloud Service Enterprise Edition 11g with Coherence, you can manually create your own Coherence cluster. Over the coming year, we plan to release a number of Coherence-related data grid and caching services. Stay tuned for more details.  For more information: cloud.oracle.com/java Users guide - Oracle Coherence with JCS New Java Cloud Service 12.1.3 - another major update to JCS since January: public blog post

This entry is reposted from the new cloud-focused blog at Oracle, "The Cloud Front." Today we are excited to announce a major product innovation - Oracle Coherence is now available as a feature of the...

Coherence

Coherence Special Interest Groups [SIGs] Coming to Asia-Pacific!

The Coherence Special Interest Groups [SIGs] are coming to APAC!Quick jump to Shenzen Dec 8 Beijing Dec 10 Seoul Dec 12 Melbourne Dec 15We've invested a lot of energy over the years to be sure that you have local or semi-local access to Coherence product management and technical staff. Coming up in this round of APAC SIGs, we have myriad topics that you'll find valuable, including showcasing some customer use patterns, advances in database connectivity, real-time data processing and much more.You'll also hear a lot about our next major release (12.2.1) and get a sneak peek at some of the new technologies we're including for developers, including utilizing Java 8 (yes, Java 8 - you heard right), and support for maximum-availability architecture [MAA] across multiple data centers.If you're in China, Korea or Australia, these are can't-miss events! We look forward to seeing you there! Shenzen, China, December 810 a.m. - 4 pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, Best Practices, Real-Time Persistence and Snapshotting, and more. Full agenda Beijing, China, December 1010 a.m. - 4:00 pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, Best Practices, Real-Time Persistence and Snapshotting, and more. Full agenda Seoul, Korea, December 129 a.m. - 4:00 pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, customer use cases, Database Synchronization, Real-Time Persistence and Snapshotting, and more. Full agenda Melbourne, Australia, December 1510 a.m. - 4:45 pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, Coherence + Java 8, Real-Time Cache Persistence and more. Full agenda

The Coherence Special Interest Groups [SIGs] are coming to APAC!Quick jump to Shenzen Dec 8 Beijing Dec 10 Seoul Dec 12 Melbourne Dec 15We've invested a lot of energy over the years to be sure that you have...

Coherence

Best Questions: 'Top Uses Cases for In-Memory Data Grids' Webcast

We held a webcast last week (now available free on-demand) on the most common use cases for in-memory data grids, and it was one of the year's most-attended online events. You all kept us busy answering your questions and introducing you to concepts that can help you solve some of your core IT challenges. We thought it'd be useful to record some of the questions and answers we received just in case you missed them (some questions lightly edited for clarity). How big is the Coherence footprint ? Does it support Virtual and physical hardware? The software library itself is pretty small, most of the footprint is for storing data. It does support both physical and virtual. Backend: which languages does it support? example: java , pojo, .net... Java, .NET, C , REST, memcached clients, JCache Clients (JSR-107) Will cache updating work with version 11gR2? I think I understand your question. You want to know how changes to the database are reflected in the cache. In Coherence 12c, we introduced GoldenGate HotCache, which listens to database change logs and pushes changes into Coherence. This will work with version 11gR2. Once data is cached on Coherence, what is the criteria to offload the data, will it be taken care of by Coherence, or we need to provide any specific logic/criteria? Offload works by accessing data in a key-based manner. If your application is SQL based and you don't want to change it, Coherence may not be a good fit you, unless you are using some persistence technology like JPA. What is a Coherence application? This was mentioned in the initial part of the presentation. By Coherence application, we mean an application that uses Coherence. Probably a bad term - some large apps are built on Coherence as if Coherence itself were the app server! From what I understood, Coherence works with multiple application servers like Oracle WebLogic. Is there any advantage to using Coherence with Oracle WebLogic as compared to JBoss/Tomcat or any other application servers? Yes, we have built-in integration for areas like lifecycle management and monitoring, application deployment, security, simpler session management, and more coming in future releases. If Cohernece is down due to some reason, will the application will be able to access the original data? If you use Coherence as a "look aside" cache, yes. But most customers do not do this and rely on Coherence being available, together with a reasonable DR and "Maximum Availability Architecture" strategy. How much data can be cached by Coherence without effecting its performance? Coherence is designed to be horizontally scalable. Customers run 100s of cluster members in some cases, and even smaller clusters can cache TBs of data with no performance impact. Do we have any reporting machanisim on the caching to get details about highly accessed data & how often? OOTB, Not on a per object basics, but at a certain higher level. We are adding auditing functionality to Coherence soon that may enable you to track that. Today, though, we have hooks where you can insert your own logic to do this. You can see the webcast in its entirety on-demand, or if you have further questions please feel free to comment here.

We held a webcast last week (now available free on-demand) on the most common use cases for in-memory data grids, and it was one of the year's most-attended online events. You all kept us busy...

Coherence

What's the Difference Between In-Memory Database and In-Memory Data Grids?

We've gotten a lot of questions recently about the differences between Oracle's announced in-memory database and in-memory data grid technologies like Coherence. Let's clear that up, with a dusted-off excerpt from our "State of the Union" blog a couple of months ago. Just in case you missed it. The In-Memory Databaseoption has led to confusion in the In-Memory Data Grid market Let us try to clarify some of the confusion. Oracle Coherence is not embedded in the In-Memory Option forthe Oracle Database. Although we haveseen rumors of this to the contrary, this would make absolutely no sense. Coherence is a distributed key-value store; the in-memory option is a columnar store. The In-Memory option is purpose built andoptimized to work as one with the database with its primary access method beingSQL. (Of course, Coherence is part ofmany other Oracle solutions, such as Oracle Service Bus which offers webservices results caching via a check box,or the Elastic Charging Engine, which is architected on top of Coherence’sfault-tolerant distributed processing.  As you can see, Coherence has becomestrategic within Oracle.)The In-Memory Option focuses on different use cases thanCoherence. In-Memory focuses on ad-hocfull-table scans of unindexed data.  In the words of Maria Colgan"The main use case for the In-Memory column store is fast analytics, whichhas very different access patterns to OLTP. Typically analytic queries access afew columns from a table but they examine all of the entries in thosecolumns. You need to scan the whole table or segment, not just a small part ofthe table to answer these queries.” Storing these columns in-memory, incolumnar format, speeds full-table scans orders of magnitude. By contrast, Coherence focuses on providing scale-out datamanagement and processing to middleware applications thereby reducing load onshared services such as databases, mainframes, and web services.  By caching data in domain object format closethe application, Coherence also increases responsiveness and availability ofapplications.  Hopefully, this helps you understand some key differences and benefits. 

We've gotten a lot of questions recently about the differences between Oracle's announced in-memory database and in-memory data grid technologies like Coherence. Let's clear that up, with a dusted-off...

Coherence

Coherence SIGs Coming to Europe in November!

The Coherence Special Interest Groups [SIGs] are coming to a town near you!Quick jump to Munich Nov 4 Stockholm Nov 5 London Nov 7 Madrid Nov 12We've invested a lot of energy over the years to be sure that you have local or semi-local access to Coherence product management and technical staff. Coming up in this round of SIGs, we have myriad topics that you'll find valuable, including showcasing the innovative usage of some of our recent 12c release features like HotCache (to keep your cache always-fresh) and Live Events for in-grid data processing. You'll also hear a lot about our next major release (12.2.1) and get a sneak peek at some of the advances we're making for developers utilizing Java 8 (yes, Java 8 - you heard right), and support for maximum-availability architecture [MAA] across multiple data centers.Finally, for customers of multiple Oracle products, we're participating in several Cloud Application Foundation SIGs that will incorporate WebLogic and Java at a holistic level. We look forward to seeing you there! Munich, November 4 [with WebLogic & Java]9 a.m. - 2:45pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, WebLogic for Java EE Multitenancy, Java 8, Cache and Database Synchronization and more. Full agenda Stockholm, November 5 [with WebLogic & Java]9 a.m. - 2:15pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, WebLogic for Java EE Multitenancy, Java 8, Cache and Database Synchronization and more. Full agenda London, November 710 a.m. - 4:45pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, Coherence + Java 8, Managing Mission-Critical Large Clusters and more. Full agenda Madrid, November 129 a.m. - 1:00pm REGISTER | DIRECTIONSAgenda topics include Coherence road maps, Coherence + Java 8, Real-Time Cache Persistence and more. Full agenda

The Coherence Special Interest Groups [SIGs] are coming to a town near you!Quick jump to Munich Nov 4 Stockholm Nov 5 London Nov 7 Madrid Nov 12We've invested a lot of energy over the years to be sure...

Coherence

Coherence Sessions to Watch For at Oracle OpenWorld 2014

Headed to Oracle OpenWorld? Here's a handful of some of our favorite Coherence sessions you won't want to miss. (Hint: for even more content, you can see the Cloud Application Foundation 'Focus On' Document for a broad view of sessions and labs available.) Coherence Roadmap and StrategyMonday, 4:00 pm, Moscone South 304This is the landmark session for everything you wanted to know about Coherence as it stands today, and our planned directions for the future. If you pick just one session, this one is key. From the content catalog:Oracle Coherence 12c, the market-leading in-memory data grid, introduced a range of new features to help customers deliver next-generation applications that scale to handle mobile, social, and Internet of Things demands. The HotCache feature of Oracle GoldenGate makes database updates available in real time to applications, and Oracle Coherence’s Managed Coherence Servers feature shortens time to market and simplifies operations. This session reviews these features and looks at the most recent features and the roadmap. Oracle Coherence provides new options for integration, including support for JCache (JSR 107), and new monitoring capabilities. The session also looks at exciting innovative features such as Recoverable Caching, Federated Caching, multitenancy, and support for Oracle Cloud. Synergy: Using WebLogic and Coherence in TandemMonday, 5:15 pm, Moscone South 236Following the strategy session just down the hall, we're excited to welcome a panel of customers and experts to chat on the synergies between Coherence and WebLogic. It makes sesnse they'd work well topgether, of course. But there may be some hidden advantages you weren't aware of if you're a customer of either product. Find out the secrets for real-world users. From the content catalog:Oracle WebLogic Server and Oracle Coherence deliver industry leadership as standalone products and are simply unbeatable in combination. In this session, the members of a panel of experts describe how they have leveraged the integration between Oracle WebLogic Server and Oracle Coherence to deliver new levels of application performance, scalability, and reliability. Lockdown! Security Practices for WebLogic and CoherenceTuesday, 5:00 pm, Moscone South 304Security is always in the news. Hacked this, stolen that. But we have some security features and best practices to make sure your apps are secure across the Oracle stack. If you're secure-minded, this is for you. From the content catalog:Securing application server deployments is more important now than it has ever been. Java security, Heartbleed, and cloud computing are topics that have driven new levels of security awareness during the past year. For Oracle specifically, Oracle Cloud offerings based on Oracle WebLogic Server and Oracle Coherence have driven new requirements for securing Oracle WebLogic/Oracle Coherence environments. This session reviews evolving security challenges, best practices Oracle has adopted for securing Oracle Cloud systems, and practical advice from the Oracle WebLogic security team for securing your own on-premises Oracle WebLogic/Oracle Coherence systems. Come to this session, and learn how to lock down your systems. Pushing Database Transactions to JCache with Coherence and GoldenGate LABWednesday, 4:15 pm, Hotel Nikko - Ballroom IIGet your hands into the code with the new JCache spec, Coherence and GoldenGate. You'll have a chance to see how the tight integration between these specs and products keeps your cache always-fresh. From the content catalog:Speeding and scaling access to changing data, in a cost-effective way, is a serious challenge. Oracle Coherence continues to reduce this challenge, with its new JCache, HotCache, and Managed Coherence Servers features. Implementing the recent JSR 107 (JCache) specification, Oracle Coherence now offers cache access through a standard API. The HotCache feature leverages Oracle GoldenGate to push database changes to Oracle Coherence in an event-driven way, and in real time. Meanwhile Managed Coherence Servers makes managing an Oracle Coherence application as simple as managing a Java EE application. In this hands-on lab, you’ll use Managed Coherence Servers to deploy and run a JCache application that receives real-time updates through HotCache as an underlying database changes. Rapid Delivery of Innovative Real-Time Apps with CoherenceThursday, 9:30am, Moscone South 304With increased data volumes, real-time processing is a must. You'll get a peek at how you can make real-tim apps a reality for your organization. From the content catalog:Oracle Coherence 12c provides innovative features and lifecycle management tools that make it well suited to quickly deliver and roll out new mobile and rich client services. This case study shows how Oracle Coherence, leveraging its Managed Coherence Servers feature and Oracle GoldenGate’s HotCache feature, is used to extend product offerings while reducing both costs and time to market. The session details the technical solution, including implementation of HotCache to provide real-time updates from the database to Oracle Coherence–based applications, plus the use of Managed Coherence Servers to deploy Oracle Coherence applications as easily as a Java EE application. It also discusses business benefits such as before-and-after metrics and implementation costs. There's much more, so don't forget to see the Coherence Focus-On Document and the Cloud Application Foundation Focus-On Document for more sessions, labs and details.

Headed to Oracle OpenWorld? Here's a handful of some of our favorite Coherence sessions you won't want to miss. (Hint: for even more content, you can see the Cloud Application Foundation 'Focus On'...

Coherence

Running Tabular Reports with Coherence 12.1.3 Reporter

Overview  With the latest release of Coherence 12.1.3, we introduced the Coherence JVisualVM Plug-in as part of the product. As part of the internal development of the plug-in, we needed a way of running "Reporter" reports and retrieving the output directly from the operation return value.  The current runReport operation (which is useful for writing out reports to disk) takes in a report name and then writes the output to the filesystem using the OutputPath defined on the MBean. Based upon this requirement, we introduced the runTabularReport operation on the Reporter MBean which takes either a report name or report XML and returns a TabularData object containing the report output. We exposed this on the MBean as we knew this would be useful for customers. MBeans  The following operations were added in 12.1.3 runTabularReport(String sReportFile) runTabularGroupReport(String sReportName, Map mapXmlReports) We are going to focus on the runTabularReport which is the easiest to understand and use. Example  Startup JVisualVM and connect to a JMX enabled cluster. Select the Reporter MBean and select one of the nodes. Enter the following report name as an argument and click runTabularReport. (Make sure you have at least one cache defined, otherwise null will be returned). reports/report-cache-size.xml   The following will be displayed. If you have more than one cache then you can scroll through the results of the report.  To view the full list of reports available out of the box you can issue the following command against coherence.jar.  timmiddleton$ jar tvf coherence.jar | grep reports     0 Thu Apr 03 14:49:54 WST 2014 reports/  2272 Thu Apr 03 14:49:52 WST 2014 reports/report-all.xml 17745 Thu Apr 03 14:49:52 WST 2014 reports/report-cache-effectiveness.xml  2629 Thu Apr 03 14:49:52 WST 2014 reports/report-cache-size.xml  2152 Thu Apr 03 14:49:52 WST 2014 reports/report-flashjournal.xml  1672 Thu Apr 03 14:49:52 WST 2014 reports/report-group.xml  1368 Thu Apr 03 14:49:52 WST 2014 reports/report-management.xml  4652 Thu Apr 03 14:49:52 WST 2014 reports/report-memory-status.xml  4541 Thu Apr 03 14:49:52 WST 2014 reports/report-network-health-detail.xml  7743 Thu Apr 03 14:49:52 WST 2014 reports/report-network-health.xml  1656 Thu Apr 03 14:49:52 WST 2014 reports/report-node.xml  2638 Thu Apr 03 14:49:52 WST 2014 reports/report-persistence-detail.xml  2929 Thu Apr 03 14:49:52 WST 2014 reports/report-persistence.xml  4270 Thu Apr 03 14:49:52 WST 2014 reports/report-proxy.xml  1715 Thu Apr 03 14:49:52 WST 2014 reports/report-ramjournal.xml  4588 Thu Apr 03 14:49:52 WST 2014 reports/report-service.xml 18473 Thu Apr 03 14:49:52 WST 2014 reports/report-web-cache-overflow.xml 18456 Thu Apr 03 14:49:52 WST 2014 reports/report-web-cache-sessions.xml  1771 Thu Apr 03 14:49:52 WST 2014 reports/report-web-group.xml  4625 Thu Apr 03 14:49:52 WST 2014 reports/report-web-service.xml 16907 Thu Apr 03 14:49:52 WST 2014 reports/report-web-sessions.xml  3856 Thu Apr 03 14:49:52 WST 2014 reports/report-web.xml This is well and good, but the main reason you would want to do this is to directly run a report given a MBeanServer connection and retrieve the results.   Programatic Access to Reports Output  All you need to run a report is an MBeanServer connection. This can be obtained via either using the MBeanHelper as per below, or there are a couple of code examples out there you could utilise such as here or here. (Thanks to Ben and Aleks) Then you can invoke the runTabularReport using that connection. In my example below, i'm doing the following: Setting system properties to ensure we have JMX management set Connect to / create a Coherence Cluster Get the MBeanServer using the helper Create some caches and insert data  Run the report Navigate the TabularData and output the values. The full code example is below.  import com.tangosol.net.CacheFactory;import com.tangosol.net.Cluster;import com.tangosol.net.NamedCache;import com.tangosol.net.management.MBeanHelper;import java.util.Collection;import java.util.Set;import javax.management.InstanceNotFoundException;import javax.management.MBeanException;import javax.management.MBeanServer;import javax.management.MalformedObjectNameException;import javax.management.ObjectName;import javax.management.ReflectionException;import javax.management.openmbean.TabularData;public class ReporterExample { public static void main(String[] args) throws InstanceNotFoundException, MalformedObjectNameException, ReflectionException, MBeanException { // enable management before we start/join a cluster System.setProperty("tangosol.coherence.management", "all"); System.setProperty("tangosol.coherence.log.level", "2"); Cluster cluster = CacheFactory.ensureCluster(); MBeanServer server = MBeanHelper.findMBeanServer(); // create caches and add some data NamedCache nc1 = CacheFactory.getCache("dist-tim1"); NamedCache nc2 = CacheFactory.getCache("dist-tim2"); nc1.put("key1", "value1"); nc2.put("key1", "value1"); // get the local member to run the report on int nLocalMemberId = cluster.getLocalMember().getId(); TabularData reportData = (TabularData) server.invoke( new ObjectName("Coherence:type=Reporter,nodeId=" + nLocalMemberId), "runTabularReport", new Object[] {"reports/report-cache-size.xml"}, new String[] {"java.lang.String"}); // navigate the data if (reportData != null) { Set<?> setKeys = reportData.keySet(); if (setKeys.size() > 0) { // loop through each key, which is actually the row of data for (Object oKey : setKeys) { // get the columns as an array Object[] aoColumns = ((Collection<Object>) oKey).toArray(); for (int i = 0; i < aoColumns.length; i++) { System.out.print(aoColumns[i].toString() + "\t"); } System.out.println(); } } } } } Running this for my cluster and two caches gives the following output:  Oracle Coherence Version 12.1.3.0.0  Grid Edition: Development modeCopyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.1 0 DistributedCache dist-tim 100000.0 3.1776896E7 30.3048095703125 317.768962 0 DistributedCache dist-tim1 1.0 224.0 2.13623046875E-4 224.03 0 DistributedCache dist-tim2 1.0 224.0 2.13623046875E-4 224.0 You can also pass in report XML rather than an actual report file, so you can build up your reporter XML on the fly or manipulate the XML before you run it, e.g. parameterise it! If you have a report group, it can be a little trickier to navigate,  but here is some guidance. Execute the specified report file or report XML defined in the sReport argument. If the report XML file or XML content defines a single report, the returned TabularData will have a CompositeData for each row of values from the report. It will also include a rowId attribute for indexing. For example:         TabularData(sReportFile) =                TabularData[ CompositeData { attribute1 -> value1,                                             attribute2 -> value2,                                             rowId      -> 1},                             CompositeData { attribute1 -> value1,                                             attribute2 -> value2,                                             rowId      -> 2} ]    If the specified file or XML content defines a report group, the returned TabularData will have a single CompositeData with report-names as keys and TabularDatas of the reports as values. For example:         TabularData[ CompositeData { report1 -> TabularData(report1),                                 report2 -> TabularData(report2) } ]    Enjoy. 

Overview  With the latest release of Coherence 12.1.3, we introduced the Coherence JVisualVM Plug-in as part of the product. As part of the internal development of the plug-in, we needed a way of...

Integration

Getting Started With The Coherence Memcached Adaptor

Overview Coherence 12c (12.1.3) adds support for Memcached clients to directly store data a in Coherence cluster using the Binary Memcached protocol. This post outlines how to configure the Coherence Memcached Adaptor and includes a simple PHP example to show how Memecached clients can connect to a Coherence cluster. The Memcached adaptor is configured as a proxy service that runs in the Coherence cluster.  This is highlighted in the example cache configuration below: <?xml version="1.0"?><cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"  xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config   http://xmlns.oracle.com/coherence/coherence-cache-config/1.2/coherence-cache-config.xsd">  <defaults>    <serializer>pof</serializer>  </defaults>  <caching-scheme-mapping>    <cache-mapping>      <cache-name>memcache</cache-name>      <scheme-name>DistributedCacheScheme</scheme-name>    </cache-mapping>  </caching-scheme-mapping>  <caching-schemes>    <distributed-scheme>      <scheme-name>DistributedCacheScheme</scheme-name>      <service-name>DistributedCacheService</service-name>      <backing-map-scheme>        <local-scheme/>      </backing-map-scheme>      <autostart>true</autostart>    </distributed-scheme>    <proxy-scheme>      <service-name>MemecachdTcpCacheService</service-name>      <acceptor-config>        <memcached-acceptor>          <cache-name>memcache</cache-name>          <memcached-auth-method>plain</memcached-auth-method>          <address-provider>memcached-addr-provider</address-provider>        </memcached-acceptor>      </acceptor-config>      <autostart>true</autostart>    </proxy-scheme>  </caching-schemes></cache-config> Note a couple of things: The <cache-name> element under <memcached-acceptor> defines the distributed cache for the Memcached acceptor.  All Memcached clients connecting to this acceptor will access this cache. You can optionally enable client authentication by setting the <memcached-auth-method>. Currently the only supported method is SASL (Simple Authentication and Security Layer) plain authentication. The <address-provider> element defines the socket-address that the Memcached Adapter will listen on for connections.  It is defined in the Coherence operational configuration file. Below is an example operational configuration file that shows the setup of an address-provider: <?xml version='1.0'?><coherence xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-operational-config  http://xmlns.oracle.com/coherence/coherence-operational-config/1.2/coherence-operational-config.xsd"> <cluster-config>    <address-providers>      <address-provider id="memcached-addr-provider">        <socket-address>          <address>127.0.0.1</address>          <port>11211</port>        </socket-address>      </address-provider>    </address-providers> </cluster-config>  <logging-config>    <destination system-property="tangosol.coherence.log">stdout</destination>    <severity-level system-property="tangosol.coherence.log.level">9</severity-level>  </logging-config>  <security-config>    <identity-asserter>      <class-name>memcached.PasswordIdentityAsserter</class-name>    </identity-asserter>  </security-config></coherence> The address provider with id memcached-addr-provider defines the socket-address reference.  Authentication For authentication, you can write a custom IdentityAsserter implementation and declare it in the operational configuration file. This is shown above but the relevant snippet is shown below: …<security-config>    <identity-asserter>      <class-name>memcached.PasswordIdentityAsserter</class-name>    </identity-asserter> </security-config>… The Memcached Adapter calls the IdentityAsserter implementation and passes the com.tangosol.net.security.UsernameAndPassword object as a token. A sample IdentityAsserter is shown below: package memcached; import com.tangosol.net.Service;import com.tangosol.net.security.IdentityAsserter;import com.tangosol.net.security.UsernameAndPassword; import javax.security.auth.Subject; public class PasswordIdentityAsserter implements IdentityAsserter {  public Subject assertIdentity(Object oToken, Service service)  throws SecurityException {    if (oToken instanceof UsernameAndPassword) {      UsernameAndPassword secToken = (UsernameAndPassword) oToken;      String sPwd = new String(secToken.getPassword());      if (secToken.getUsername().equals("username")          && sPwd.equals("password")) {        return new Subject();      }    }    throw new SecurityException("Access denied");  }} PHP Memcached client There are 2 very popular PHP memcached clients – Memcache and Memcached. Memcached is the newer one and supports both the Binary Memcached protocol and SASL authentication. To enable Memcached and SASL, add the following to the php.ini file:     extension=memcached.so    memcached.use_sasl = 1 The following PHP code uses the PHP Memcached client library to connect to a Coherence cluster, add an entry and then retrieve it again. <?php  $m = new Memcached();  $m->setOption(Memcached::OPT_BINARY_PROTOCOL, true);  $m->setSaslAuthData("username", "password");  if (!count($m->getServerList())) {    // List of Coherence proxy service nodes    $m->addServer('127.0.0.1', 11211);  }  print_r($m->getVersion());  $m->set('test', 'test_string');  print_r('Cache value: ' . $m->get('test'));?> Putting it all together This is easy to try out for yourself.  First download and install Coherence, PHP and the PHP Memcached client library. Then in a test dir create an operational configuration file called tangosol-coherence-override.xml with the operational configuration details outlined above. Similarly create a cache configuration file called coherence-cache-config.xml with the specified cache configuration for Memcached clients. Finally, in the test dir create a dir called memcached and in it create a file called PasswordIdentityAsserter.java with the code listing above. On the PHP side just save the example PHP code above to a file called test.php. Now we are ready to test everything.  To startup Coherence we will need to refer to the Coherence libraries. To make this easier define an environment variable for the location of the Coherence installation called COH_HOME. Below we will assume a Linux/OSX environment but its very similar on Windows. Build the PasswordIdentityAsserter by running the following command in your test dir: javac –cp $COH_HOME/lib/coherence.jar memcached/PasswordIdentityAsserter.java Now start a Coherence cache server to store the Memcached client data: java -cp .:$COH_HOME/lib/coherence.jar com.tangosol.net.DefaultCacheServer Lastly test that the PHP client can connect, store and read data securely via the Coherence Memcached Adaptor: php test.phpThe output should be: Array (     [127.0.0.1:11211] => 12.1.3 ) Cache value: test_string This shows that the value inserted into Coherence by the PHP client, test_string, has successfully been read back.

Overview Coherence 12c (12.1.3) adds support for Memcached clients to directly store data a in Coherence cluster using the Binary Memcached protocol. This post outlines how to configure the Coherence...

Coherence

Coherence State of the Union

Six years after I joined Coherence following Oracle’sacquisition of BEA, it is great to see how the industry has embraced In-MemoryData Grid’s as a mainstream technology to scale middleware applications. This adoption is driven by growth in customerdemand to manage more and more data, and to act on that data faster andfaster. The mission that the Coherenceteam set upon over a decade ago has come to fruition, and our core team – stillintact all this time later even as we continue to grow the team in newdirections – is working on new challenges as in-memory data grid basedsolutions become standard. The latest release of Coherence and the new announcement ofthe In-Memory Option for the Oracle Database make this an opportune time toanswer some recent questions I’ve received about where Coherence is headed. First lets review the recentannouncements. In June we announced Coherence 12.1.3, thesecond release of Coherence 12cR1. Coherence 12.1.2, released in 2013, introduced groundbreakingoperational improvements like ManagedCoherence Servers and GAR files, GoldenGate HotCache to keep caches anddatabases in sync, and Live Events to respond to state and data changes inreal-time. With Coherence 12.1.3 Oraclebecame the first commercial vendor to support the recently ratified JSR-107(JCache) spec, the standard for caching on the Java platform. We also introduced a JVisualVM plugin fordevelopers, asynchronous EntryProcessors to allow clients to submit work to thedata grid without waiting for a response, memcached client support, andoptimization improvements to our Exalogic Infiniband optimizations(Exabus). Check out all the cool newfeatures of Coherence 12.1.3 in our release notes. In July, Oracleannounced the availability of the In-MemoryOption for the Oracle Database. TheIn-Memory Option is designed to accelerate database performance by orders ofmagnitude for analytics, data warehousing, and reporting by storing columnardata in-memory. It supports ad-hoc queriesand full-table scans without requiring application changes; the interface isstandard SQL. Maria Colgan answersfrequently asked questions about this release on her blog. The new In-Memoryoption has also led to some confusion in the In-Memory Data Grid market. Let me try to clarify some of the confusion. Oracle Coherence is not embedded in the In-Memory Option forthe Oracle Database. Although I haveseen rumors of this to the contrary, this would make absolutely no sense. Coherence is a distributed key-value store; the in-memory option is a columnar store. The In-Memory option is purpose built andoptimized to work as one with the database with its primary access method beingSQL. (Of course, Coherence is part ofmany other Oracle solutions, such as Oracle Service Bus which offers webservices results caching via a check box,or the Elastic Charging Engine, which is architected on top of Coherence’sfault-tolerant distributed processing.  (As you can see, Coherence has becomestrategic within Oracle.) The In-Memory Option focuses on different use cases thanCoherence. In-Memory focuses on ad-hocfull-table scans of unindexed data.  in the words of Maria Colgan"The main use case for the In-Memory column store is fast analytics, whichhas very different access patterns to OLTP. Typically analytic queries access afew columns from a table but they examine all of the entries in thosecolumns. You need to scan the whole table or segment, not just a small part ofthe table to answer these queries.” Storing these columns in-memory, incolumnar format, speeds full-table scans orders of magnitude. By contrast, Coherence focuses on providing scale-out datamanagement and processing to middleware applications thereby reducing load onshared services such as databases, mainframes, and web services.  By caching data in domain object format closethe application, Coherence also increases responsiveness and availability ofapplications.  So if the In-memoryOption is not related to Coherence, what has the Coherence team been up to? Glad you asked! Aside from the two 12c releases over the pastyear, we continue to invest in several strategic areas (you can get a prettygood sneak peak of upcoming releases at Oracle OpenWorld next month). The Cloud. Oracle has increased investment inaccelerating’s Coherence role in our various cloud offerings, from public toprivate PAAS. Our Coherence cloud teamis leveraging investments in Managed Coherence Servers to simplify operationsand continue to integrate Coherence into the cloud environment. In addition, Coherence integrates with otherOracle technology to provide a full cloud management solution. OperationalEfficiency. Related to the Cloud,but somewhat separate, we are innovating to reduce the cost it takes, both froma hardware and IT staff perspective, to manage an in-memory data grid solution. I Maximum AvailabilityArchitectures.  Several features inour next release will dramatically increase the availability of data to yourapplications. Features include newadvancements in mutl-datacenter solutions and protecting the availability ofthe in-memory data grid within a single datacenter. DeveloperProductivity.  We continue to investin new functionality, new integration points (e.g. REST, JSR-107, memcached inrecent releases) that complement our native clients. We also invest in new product features thatsimplify the lives of developers both from an API (AsynchronousEntryProcessors) and from a tooling (JVisual VM plugin and Coherence tooling inOracle Enterprise Pack for Eclipse, or OEPE). To get more specific of all these investments, you can joinme at my Coherence Product Strategy and Roadmap talk at OpenWorld.

Six years after I joined Coherence following Oracle’s acquisition of BEA, it is great to see how the industry has embraced In-MemoryData Grid’s as a mainstream technology to scale...

Coherence

NEW: Oracle Coherence 12.1.3

The latest version Coherence 12c (12.1.3)  has just become available on OTN. This release introduces some significant new features, like support for Memcached clients and the new JCache/JSR107 Java Caching Standard. It also contains the new JVisualVM monitoring plug-in and has some cool optimisations.  Support for Memcached, through the Memcached Adaptor, now allows clients written in PHP, Ruby, Python and other technologies to directly access data in Coherence. It also enables Memcached clients to store larger entries than before and and store them resiliently. The new JCache Adaptor provides support for the Java Caching API, JSR107, and comes hot on the heels of the standards ratification. JCache provides a vendor neutral mechanism for using caching in Java applications, and as you would expect the Coherence implementation is fully compliant with the standard for complete portability. The Coherence JVisual VM plug-in provides real-time Coherence monitoring and now comes bundled with Coherence itself. It aggregates and visualises key metrics across a Coherence cluster, displaying them in graphs and tables that are easy to understand. It also provides the ability to export metrics and displays visual alerts when certain thresholds have been reached. Finally, there is a wealth of optimisations and improvements, like asynchronous entry processors to make asynchronous processing easier and the deserialisation accelerator, for boosting query performance. You can find out more about these features in subsequent blog postings and the feature links above for YouTube feature screen casts. Further details about other new features in the 12.1.3 release of the Oracle Cloud Application Platform (CAF) can also be found here. 

The latest version Coherence 12c (12.1.3)  has just become available on OTN. This release introduces some significant new features, like support for Memcached clients and the new JCache/JSR107 Java...

Coherence

Webcast On-Demand: Building Java EE Apps That Scale

With some awesome work by one of our architects, Randy Stafford, we recently completed a webcast on scaling Java EE apps efficiently. Did you miss it? No problem. We have a replay available on-demand for you. Just hit the '+' sign drop-down for access.Topics include: Domain object caching Service response caching Session state caching JSR-107 HotCache and more! Further, we had several interesting questions asked by our audience, and we thought we'd share a sampling of those here for you - just in case you had the same queries yourself. Enjoy! What is the largest Coherence deployment out there?We have seen deployments with over 500 JVMs in the Coherence cluster, and deployments with over 1000 JVMs using the Coherence jar file, in one system. On the management side there is an ecosystem of monitoring tools from Oracle and third parties with dashboards graphing values from Coherence's JMX instrumentation. For lifecycle management we have seen a lot of custom scripting over the years, but we've also integrated closely with WebLogic to leverage its management ecosystem for deploying Coherence-based applications and managing process life cycles. That integration introduces a new Java EE archive type, the Grid Archive or GAR, which embeds in an EAR and can be seen by a WAR in WebLogic. That integration also doesn't require any extra WebLogic licensing if Coherence is licensed. How is Coherence different from a NoSQL Database like MongoDB?Coherence can be considered a NoSQL technology. It pre-dates the NoSQL movement, having been first released in 2001 whereas the term "NoSQL" was coined in 2009. Coherence has a key-value data model primarily but can also be used for document data models. Coherence manages data in memory currently, though disk persistence is in a future release currently in beta testing. Where the data is managed yields a few differences from the most well-known NoSQL products: access latency is faster with Coherence, though well-known NoSQL databases can manage more data. Coherence also has features that well-known NoSQL database lack, such as grid computing, eventing, and data source integration. Finally Coherence has had 15 years of maturation and hardening from usage in mission-critical systems across a variety of industries, particularly financial services. Can I use Coherence for local caching?Yes, you get additional features beyond just a java.util.Map: you get expiration capabilities, size-limitation capabilities, eventing capabilites, etc. Are there APIs available for GoldenGate HotCache?It's mostly a black box. You configure it, and it just puts objects into your caches. However you can treat it as a glass box, and use Coherence event interceptors to enhance its behavior - and there are use cases for that. Are Coherence caches updated transactionally?Coherence provides several mechanisms for concurrency control. If a project insists on full-blown JTA / XA distributed transactions, Coherence caches can participate as resources. But nobody does that because it's a performance and scalability anti-pattern. At finer granularity, Coherence guarantees strict ordering of all operations (reads and writes) against a single cache key if the operations are done using Coherence's "EntryProcessor" feature. And Coherence has a unique feature called "partition-level transactions" which guarantees atomic writes of multiple cache entries (even in different caches) without requiring JTA / XA distributed transaction semantics.

With some awesome work by one of our architects, Randy Stafford, we recently completed a webcast on scaling Java EE apps efficiently. Did you miss it? No problem. We have a replay available on-demand...

Coherence Community

New: VisualVM Plugin for Coherence - released through the Coherence Incubator

We are excited to announce the availability of the Coherence VisualVM developer plug-in which is part of the new Coherence Incubator 12.1.0 release. The community plug-in allows developers to view summarised Coherence related statistics and information when connected to a JMX enabled Coherence cluster.  Information is collected and summarised from the various Coherence MBeans and displayed in a tabular and graphical format. You also have the ability to select various services and caches to display more detailed node level information about each. Once installed you will see an additional 'Coherence' tab, when you connect to your Coherence process which is running an MBean server. If you select of this tab, a 'Cluster Summary' tab will be displayed. Features The number of sub-tabs displayed varies depending upon what features you are using, but can include: Cluster Overview - High-level information about the Coherence cluster including cluster name, version, member count and 'Cluster StatusHA'. Summary graphs show total cluster memory available and used, packet publisher and receiver success rates and load averages for machines running Coherence. Machines - A list of the physical machines that make up the Coherence cluster as well as information about the load averages and available memory on these machines. Members - The full list of Coherence members/nodes, including individual publisher/receiver success rates, memory and send queue sizes. Services - Information about the running services including partition counts and statusHA values. If you select a service, on the next data refresh you will see detailed thread information for each node of the service as well as graphs of that information. Caches - Displays information about any caches, including their size and memory usage information. To get the correct information to be displayed for memory usage, you must be using the binary unit-calculator. If you select a cache, on the next data refresh you will see detailed information about each node hosting that service and cache. Proxy Servers - If your cluster is running proxy servers, this tab displays information about the proxy servers and the number of connections across each proxy server and total connections. You also have the ability to include or exclude the NameService. Coherence*Web - If your cluster is configured for Coherence*Web, this tab displays information about the number of applications deployed, the number of HTTP sessions being stored as well as other information regarding session reaping. A sample of the Members sub-tab is shown below, with visual clues - that are included on the various screens - to draw attention to potential issues. The Services sub-tab below shows how to select an individual service and monitor detailed metrics, like as thread utilisation. Getting the Plug-in The plug-in is available from GitHub which hosts the source for the latest version of the Coherence Incubator (12.1.0).  The Incubator uses the Common Development and Distribution License (CDDL) license and the documentation, which includes how to build and install the plug-in, can be found here.   A pre-built NetBeans Module (NBM) is available via search.maven.org or you can build as part of the Incubator above. The VisualVM tool is available with any download of JDK 1.7 or at http://visualvm.java.net. The Coherence plug-in is designed to work with JDK 7 Update 40 and above.  If your cluster is running on an earlier version of Java, you can still use the latest JVisualVM and connect remotely to your cluster to view statistics. Note: This is a developer tool for viewing information about a single cluster at a time. Other monitoring tools such as Oracle's Enterprise Manager or other 3rd party tools are suited for enterprise level monitoring, management and alerting. 

We are excited to announce the availability of the Coherence VisualVM developer plug-in which is part of the new Coherence Incubator 12.1.0 release. The community plug-in allows developers to view...

Coherence

Hadoop MapReduce and Coherence - A Perfect Match

Hadoop MapReduce(M/R) is the most popular programming model for processing large data sets with a parallel, distributed algorithm on an HDFS cluster. Coherence is the market leading In-Memory Data Grid. While Hadoop works fine for large processingoperations, i.e. requiring many TB of data, that can be processed in a batchlike way, there are use cases where theprocessing requirements are more real-time and the data volumes are smaller,where Coherence is a better choice than HDFS for storing the data. So while I was trying toget familiar with Hadoop M/R , I realized it should not be too difficult tocombine the two. The solution presentedbelow suggests using the standard Hadoop M/R API to process data stored in a Coherence data grid as if it was storedon an HDFS cluster. With that the Hadoop WordCount example could look somethinglike: import com.oracle.coherence.mapreduce.JobClient; ...  JobConfconf = new JobConf(WordCount.class);             conf.setJobName("coherence-wordcount"); conf.setOutputKeyClass(Text.class);      conf.setOutputValueClass(IntWritable.class); conf.setMapperClass(Map. font-family: Consolas; line-height: 70%;">class); conf.setCombinerClass(Reduce.class); conf.setReducerClass(Reduce.class);      conf.setInputFormat(CoherenceInputFormat.class); conf.setOutputFormat(CoherenceOutputFormat.class); CoherenceInputFormat.setInputPaths(conf,new Path("coherence://wordcount-Input"); CoherenceOutputFormat.setOutputPath(conf,new Path(">"coherence://wordcount-Result"));   RunningJob job = JobClient.runJob(conf); The only change we made to the original Hadoopexample,  was to use CoherenceInputFormat and CoherenceOutputFormat classes and  point the input andoutput paths to Coherence caches instead of HDFS. Everything else is identicalto the Hadoop M/R API, including the user-defined Map and Reduce classes. To run this M/R job, there is no need to setup and run a Hadoop cluster, all you need is a Coherence cluster. Under the Hood At first, Entry Processors andAggregators looked like the natural choice for implementing the mapper and reducer. However both require a single member/thread to process the results, thus limiting the applicability of the solution to those cases where the mapper/aggregator  result sets can fit into a single member heap (see ChrisitanFelde excellent blog on Coherence M/R). To work around this limitation, the solution uses Distributed Invocation Services with member affinity to implement the Mapper and Reducer functionality. From there, it was just a matter of implementing a coupleof invocation services and a few Hadoop interfaces on top of Coherence caches.  Fig-1 JobClient In Hadoop, JobClient is the primary interface for the user-jobto interact with the cluster. JobClient provides facilities to submit jobs, track theirprogress, etc. Normally the user creates theapplication, describes various facets of the job via JobConf and thenuses the JobClient tosubmit the job and monitor its progress. The solution providesit’s own implementation of the JobClient Hadoop class. It expectsthat the input data for the “Map” phase is already stored on a CoherenceInputCache. JobClient delegates the work to the RunningJob class which' orchestrates the execution of the Mapper and Reducer services.Both the Mapper and theReducer communicate progress and status information through the JobInfo Coherencecache.  MapperService The MapperService is implemented as a Coherence Async InvocationService, running on allstorage enabled members of the Coherence cluster. To implement In-Situ processing of data, each member running the MapperService, uses the CoherenceRecordReader to processesonly InputCache entries stored on that member. A MemberPartitionsListener is used to keep track of the ParitionSet per cluster member and a PartitionFilter to retrieve only therelevant local entries. <distributed-scheme>             <scheme-name>dist-default</scheme-name> <partition-listener>               <class-name>com.oracle.coherence.mapreduce.listeners.MemberPartitionsListener</class-name>        </partition-listener> Fig-2 The MapperService is multi-threaded, and each thread executes the MapperTask to process a  subset of the entries stored on that member. The MapperTask invokes the user-defined “Map” class for each InputCache entry, passing it a Coherence basedimplementation of a Hadoop OutputCollector. The OutputCollector buffers the entries emitted by the Mapper class in memory, and flushes those to the MapperOutputCachewhen it reaches a predefined size. The “MapperOutputCache”is configured with Parittion Affinity sothat all entries with the same emitted key are stored in the same partition.Having partition affinity, there is no need to reshuffle the data beforerunning the Reducer. The OutputCollector also creates an entry foreach unique emitted key in the MapperUniqueKeysCache. As with the MapperCache, theMapperUniqueKeysCache has PartitionAffinity defined with the same association key, so that a master entry and allit’s associated MapperCache entries will be stored on the same coherencepartition When auser-defined “Combine” class is configured for the job, the OutputCollectorwill locally apply the “Combine” class before flushing the buffer to the MapperCache,thus reducing the number of entries that will be processed by the reducer. ReducerService The ReducerService is implemented as a Coherence Async InvocationService, running on allstorage enabled members of the cluster. Each instance of the service onlyreduces intermediate cache entries that are locally stored on the memberrunning the service instance. The ReducerService iterates through the list of unique keys emitted by the Mapper to the MapperUniqueKeysCache. For each unique key, we use a KeyAssiciatedFilterto find all MapperOutputCache entries having the same unique key. The result set is then passed to the user-defined “Reduce” class, together with a coherence based implementation of a Hadoop OutputCollectorclass. The “Reduce” class then uses the OutputCollectorto (transparently) emit its result to theCoherence ResultCache. Fig-3 Installing the demo The demo solution  ships with 2 samples,  WordCount and StandardDeviation . To run the samples: Download the solution and unzip it. The solution includes all the dependencies jars (hadoop, coherence, etc..) in the lib directory Edit the setEnv.cmd and modify the relevant environment variables. Execute the cache-server.cmd script to start one or more Coherence cache servers WordCount Sample  The WordCount sample. is the most popular MapReduce. The sample first  popules the cache with about 100,000 lines of text. The mapper then tokenize the lines and emmits a key value pair with the word as the key and a value of 1. The reducer then aggregates all the 1 for the same word To run the sample, execute the run-wordcount.cmd 2013-11-13 23:27:10.868/16.038 Oracle Coherence GE 3.7.1.1 <Info> (thread=Invocation:InvocationService, member=5): CoherenceMapReduce:JobId=1660803 JobClient:ReducerObserver - Member 1 Completed :  Machine=MBENOLIE-IL Process=8148 Job Completed **** Job Summary: **** Job Id: 1660803 **** Job Name: wordcount **** Mapper **** ------------Input Cache Size: 100000 ------------Duration: 4036 ms **** Reducer **** ------------Intermediate Cache Size: 162164 ------------Unique Entries Size: 11108 ------------Duration: 3964 ms Standard Deviation Sample The standard deviation is one of the popsample uses Mapreduce to compute the standard deviation of a sample, based on the formula in fig-4. Fig-4 For each value in the sample, the Mapper  emits a triple consisting of the value itself X, X2 and the number 1. The reducer/Combiner  adds up the emitted values and produce the result entries : N,sum(X2),(sum(X))2  To run the sample , run the run-stddev.cmd script. The following output is produced. 2013-11-13 23:44:55.818/6.953 Oracle Coherence GE 3.7.1.1 (thread=Invocation:InvocationService, member=6): CoherenceMapReduce:JobId=114742004 JobClient:ReducerObserver - Member 1 Completed : Machine=MBENOLIE-IL Process=8148 Final Calculation : entries in outputcache=3 Count = 20000.0 Sum = 1327351.1950964248 Sum of Sqrt = 8.83890996020529E7 Job Completed Standard deviation= 3.8474319265580275 **** Job Summary: **** Job Id: 114742004 **** Job Name: StandardDeviation **** Mapper **** ------------Input Cache Size: 20000 ------------Duration: 313 ms **** Reducer **** ------------Intermediate Cache Size: 60 ------------Unique Entries Size: 3 ------------Duration: 51 ms Tracking the M/R jobs  A a very primitive console is provided. Execute the console-mapreduce.cmd script, to launch the console. you can use it to track jobs progress  and display  the job  results (i.e the list of words that were counted).  Summary Leveraging existing powerful Coherence functionality, the solution combines the Real-Time/Fast-Data nature of Coherence In-Memory Data Grids with the popular Hadoop MapReduce API., giving developers the freedom to choose the most appropriate data-store while preserving their existing M/R assets.

Hadoop MapReduce(M/R) is the most popular programming model for processing large data sets with a parallel, distributed algorithm on an HDFS cluster. Coherence is the market leading In-Memory Data Grid....

Coherence

Reminder: Free, Global, Virtual Developer Day November 5th

Just a quick reminder about the FREE virtual developer day focused on Coherence (and WebLogic) coming on November 5th.This day, with content tailored for developers, will guide you through tooling updates and best practices around creating applications with WebLogic and Coherence as target platforms. We'll also explore advances in how you can manage your build, deploy and ongoing management processes to streamline your application's life cycle. And of course, we'll conclude with some hands-on labs that ensure this isn't all a bunch of made-up stuff - get your hands dirty in the code!November 5, 20139am PT/12pm ETREGISTER NOW We're offering two tracks for your attendance, though of course you're free to attend any session you wish. The first will be for pure developers with sessions around developing for WebLogic with HTML5, processing live events with Coherence, and looking at development tooling. The second is for developers who are involved in the building and management processes as part of the application life cycle. These sessions focus on using Maven for builds, using Chef and Puppet for configuration and more.We look forward to seeing you there - don't forget to invite a friend!

Just a quick reminder about the FREE virtual developer day focused on Coherence (and WebLogic) coming on November 5th.This day, with content tailored for developers, will guide you through tooling...

Coherence

Query optimization using composite indexes

Many times, during the process of creating a new Coherence application, developers do not pay attention to the way cache queries are constructed; they only check that these queries comply with functional specs. Later, performance testing shows that these perform poorly and it is then when developers start working on improvements until the non-functional performance requirements are met. This post describes the optimization process of a real-life scenario, where using a composite attribute index has brought a radical improvement in query execution times.  The execution times went down from 4 seconds to 2 milliseconds! E-commerce solution based on Oracle ATG – Endeca In the context of a new e-commerce solution based on Oracle ATG – Endeca, Oracle Coherence has been used to calculate and store SKU prices. In this architecture, a Coherence cache stores the final SKU prices used for Endeca baseline indexing. Each SKU price is calculated from a base SKU price and a series of calculations based on information from corporate global discounts. Corporate global discounts information is stored in an auxiliary Coherence cache with over 800.000 entries. In particular, to obtain each price the process needs to execute six queries over the global discount cache. After the implementation was finished, we discovered that the most expensive steps in the price calculation discount process were the global discounts cache query. This query has 10 parameters and is executed 6 times for each SKU price calculation. The steps taken to optimise this query are described below; Starting point Initial query was: String filter = "levelId = :iLevelId AND  salesCompanyId = :iSalesCompanyId AND salesChannelId = :iSalesChannelId "+ "AND departmentId = :iDepartmentId AND familyId = :iFamilyId AND brand = :iBrand AND manufacturer = :iManufacturer "+ "AND areaId = :iAreaId AND endDate >=  :iEndDate AND startDate <= :iStartDate"; Map<String, Object> params = new HashMap<String, Object>(10); // Fill all parameters. params.put("iLevelId", xxxx); // Executing filter. Filter globalDiscountsFilter = QueryHelper.createFilter(filter, params); NamedCache globalDiscountsCache = CacheFactory.getCache(CacheConstants.GLOBAL_DISCOUNTS_CACHE_NAME); Set applicableDiscounts = globalDiscountsCache.entrySet(globalDiscountsFilter); With the small dataset used for development the cache queries performed very well. However, when carrying out performance testing with a real-world sample size of 800,000 entries, each query execution was taking more than 4 seconds. First round of optimizations The first optimisation step was the creation of separate Coherence index for each of the 10 attributes used by the filter. This avoided object deserialization while executing the query. Each index was created as follows: globalDiscountsCache.addIndex(new ReflectionExtractor("getXXX" ) , false, null); After adding these indexes the query execution time was reduced to between 450 ms and 1s. However, these execution times were still not good enough.  Second round of optimizations In this optimisation phase a Coherence query explain plan was used to identify how many entires each index reduced the results set by, along with the cost in ms of executing that part of the query. Though the explain plan showed that all the indexes for the query were being used, it also showed that the ordering of the query parameters was "sub-optimal".  Parameters associated to object attributes with high-cardinality should appear at the beginning of the filter, or more specifically, the attributes that filters out the highest of number records should be placed at the beginning. But examining corporate global discount data we realized that depending on the values of the parameters used in the query the “good” order for the attributes was different. In particular, if the attributes brand and family had specific values it was more optimal to have a different query changing the order of the attributes. Ultimately, we ended up with three different optimal variants of the query that were used in its relevant cases: String filter = "brand = :iBrand AND familyId = :iFamilyId AND departmentId = :iDepartmentId AND levelId = :iLevelId "+ "AND manufacturer = :iManufacturer AND endDate >= :iEndDate AND salesCompanyId = :iSalesCompanyId "+ "AND areaId = :iAreaId AND salesChannelId = :iSalesChannelId AND startDate <= :iStartDate"; String filter = "familyId = :iFamilyId AND departmentId = :iDepartmentId AND levelId = :iLevelId AND brand = :iBrand "+ "AND manufacturer = :iManufacturer AND endDate >=  :iEndDate AND salesCompanyId = :iSalesCompanyId "+ "AND areaId = :iAreaId  AND salesChannelId = :iSalesChannelId AND startDate <= :iStartDate"; String filter = "brand = :iBrand AND departmentId = :iDepartmentId AND familyId = :iFamilyId AND levelId = :iLevelId "+ "AND manufacturer = :iManufacturer AND endDate >= :iEndDate AND salesCompanyId = :iSalesCompanyId "+ "AND areaId = :iAreaId AND salesChannelId = :iSalesChannelId AND startDate <= :iStartDate"; Using the appropriate query depending on the value of brand and family parameters the query execution time dropped to between 100 ms and 150 ms. But these these execution times were still not good enough and the solution was cumbersome. Third and last round of optimizations The third and final optimization was to introduce a composite index. However, this did mean that it was not possible to use the Coherence Query Language (CohQL), as composite indexes are not currently supporte in CohQL. As the original query had 8 parameters using EqualsFilter, 1 using GreaterEqualsFilter and 1 using LessEqualsFilter, the composite index was built for the 8 attributes using EqualsFilter. The final query had an EqualsFilter for the multiple extractor, a GreaterEqualsFilter and a LessEqualsFilter for the 2 remaining attributes.  All individual indexes were dropped except the ones being used for LessEqualsFilter and GreaterEqualsFilter. We were now running in an scenario with an 8-attributes composite filter and 2 single attribute filters. The composite index created was as follows: ValueExtractor[] ve = { new ReflectionExtractor("getSalesChannelId" ), new ReflectionExtractor("getLevelId" ),    new ReflectionExtractor("getAreaId" ), new ReflectionExtractor("getDepartmentId" ),    new ReflectionExtractor("getFamilyId" ), new ReflectionExtractor("getManufacturer" ),    new ReflectionExtractor("getBrand" ), new ReflectionExtractor("getSalesCompanyId" )}; MultiExtractor me = new MultiExtractor(ve); NamedCache globalDiscountsCache = CacheFactory.getCache(CacheConstants.GLOBAL_DISCOUNTS_CACHE_NAME); globalDiscountsCache.addIndex(me, false, null); And the final query was: ValueExtractor[] ve = { new ReflectionExtractor("getSalesChannelId" ), new ReflectionExtractor("getLevelId" ),    new ReflectionExtractor("getAreaId" ), new ReflectionExtractor("getDepartmentId" ),    new ReflectionExtractor("getFamilyId" ), new ReflectionExtractor("getManufacturer" ),    new ReflectionExtractor("getBrand" ), new ReflectionExtractor("getSalesCompanyId" )}; MultiExtractor me = new MultiExtractor(ve); // Fill composite parameters.String SalesCompanyId = xxxx;...AndFilter composite = new AndFilter(new EqualsFilter(me,                   Arrays.asList(iSalesChannelId, iLevelId, iAreaId, iDepartmentId, iFamilyId, iManufacturer, iBrand, SalesCompanyId)),                                     new GreaterEqualsFilter(new ReflectionExtractor("getEndDate" ), iEndDate)); AndFilter finalFilter = new AndFilter(composite, new LessEqualsFilter(new ReflectionExtractor("getStartDate" ), iStartDate)); NamedCache globalDiscountsCache = CacheFactory.getCache(CacheConstants.GLOBAL_DISCOUNTS_CACHE_NAME); Set applicableDiscounts = globalDiscountsCache.entrySet(finalFilter);      Using this composite index the query improved dramatically and the execution time dropped to between 2 ms and  4 ms.  These execution times completely met the non-functional performance requirements :). It should be noticed than when using the composite index the order of the attributes inside the ValueExtractor was not relevant.

Many times, during the process of creating a new Coherence application, developers do not pay attention to the way cache queries are constructed; they only check that these queries comply with...

Coherence

Spotlight on an ACE: René van Wijk

Oracle ACEs and Oracle ACE Directors are known for their strong credentials as Oracle community enthusiasts and advocates, with candidates nominated by anyone in the Oracle Technology and Applications communities. They provide not only external expertise on Oracle products, but also real-world examples and implementations of the Oracle stack, including many experts on Coherence and WebLogic. One such expert is René van Wijk, who calls the Netherlands home, and is an active member of the ACE community, and works regularly with Coherence. René is a graduate of the Delft University of Technology. Since then he has been with working with much of the Oracle stack (including WebLogic and Coherence), and is also an expert in many open standard frameworks. René transfers his knowledge and experience regularly through training, publications and presentations at Oracle-related seminars and conferences. René, most importantly, likes to relax; travel (for leisure and take photos), watch movies, listen to some fine music, and enjoy nature hikes.Looking for some Coherence insight from René? You'll find a ton of resources on his site, Middleware Magic. Follow René on Twitter: @MiddlewaremagicWant more information on becoming an Oracle ACE? See how this recognition can benefit you, and the community.

Oracle ACEs and Oracle ACE Directors are known for their strong credentials as Oracle community enthusiasts and advocates, with candidates nominated by anyone in the Oracle Technology and Applications...

Coherence

From Mainframe to Coherence: Is it worth it?

OP-Pohjola Group is theleading financial services group in Finland. It is made up of some200 member cooperative banks and OP-Pohjola Group Central Cooperativewhich they own, including its subsidiaries and closely related companies. OP-PohjolaGroup has more than four million customers -- joint banking and non-lifeinsurance customers total over one million. With over 530 branches,the Group boasts the broadest customer base and the most extensive branchnetwork in Finland. OP-Pohjola architecture is based on innovativeJava EE technology running on Oracle WebLogic Server. A vital part of their architectureis Oracle Coherence distributed cache which has enabled remarkable cost savingsin mainframe request volumes and improved performance in most critical andpopular eBanking services. Oracle Coherence has had a critical role in enablingmobile eBanking solution as a cost effective platform to build new services withexponential usage growth. Efficient caching technology has enabled both serviceefficiency and remarkable savings in mainframe costs with mainframe request volumes dropping by 40%. Thismainframe optimization has resulted in both financial cost savings whilesimultaneously enabling growth of eBanking service volumes. Comejoin OP-Pohjola in this session to hear about how this customer leverages Oracletechnologies such as WebLogic and Coherence to achieve such fantastic results. Inaddition, while at OpenWorld don’t miss other Cloud Application FoundationInnovator. You can join thesession whether you are an OpenWorld attendee or not.

OP-Pohjola Group is the leading financial services group in Finland. It is made up of some 200 member cooperative banks and OP-Pohjola Group Central Cooperativewhich they own, including its...

Coherence

Join Emerson Network Power at OpenWorld discuss Coherence for Real Time Data Processing

Emerson (NYSE: EMR), based inSt. Louis, Missouri, is a global leader in bringing technology and engineeringtogether to provide innovative solutions for customers in industrial,commercial, and consumer markets around the world. The company is comprised offive business segments: Process Management, Industrial Automation, NetworkPower, Climate Technologies, and Commercial & Residential Solutions. Salesin fiscal 2012 were $24.4 billion. Emerson Network Power delivers software,hardware and services that maximize availability, capacity and efficiency fordata centers, healthcare, and industrial facilities. A trusted industry leaderin smart infrastructure technologies, Emerson Network Power provides innovativedata center infrastructure management solutions that bridge the gap between ITand facility management and deliver efficiency and uncompromised availabilityregardless of capacity demands. Emerson Network Power’s acquisition of Avocent afew years back led to significant growth of the company’s product portfolio andcustomer base. The acquisition also provided a unique challenge for Emerson, asmany of the newly acquired products were built on open source software and weredifficult to develop, deploy and manage. Another big issue was integrating allthese point-solution products into an integrated suite. In addition, there werescalability and performance concerns. Oracle Fusion Middleware provided Emersonwith a more reliable, scalable and integrated infrastructure. As the data centerinfrastructure management (DCIM) market matures overthe next few years, Emerson believes that customers will need a lot morefunctionality than they currently realize. Moving forward, customers will needthe ability to analyze data, create insights and develop predictivecapabilities across the entire infrastructure in real-time. A single, openplatform that incorporates hardware and software to meet current and futurecustomer needs in an effective and efficient manner was a major requirement.The most critical customer needs are data scalability, availability and highperformance in processing of the real-time data. The Trellis platformrepresents a groundbreaking DCIM innovation from Emerson Network Power – onethat mitigates IT risk while increasing operational  efficiency. Comejoin Emerson in this session to hear about how this customer leverages Oracletechnologies such as Coherence, ADF, CEP, EJB and TopLink and how the OracleCoherence solution is implemented to satisfy the scalability and highperformance of real-time data processing. Also, don’t missother Cloud Application Foundation Innovators. You can join this session whether youare an OpenWorld attendee or not.

Emerson (NYSE: EMR), based in St. Louis, Missouri, is a global leader in bringing technology and engineering together to provide innovative solutions for customers in industrial,commercial, and...

Coherence

Resource: Planning your Successful Coherence Deployment

"How do I take my Coherence installation out of a concept stage and into production?"We often hear questions similar to this from customers, so we're responding with a free whitepaper that explores that very process. Of course, there's no cookie-cutter answer since your infrastructure is uniquely set up to meet your hardware and software needs. But, we do have a lot of best practices that can save you a ton of time and heartache. Oracle Coherence 12c WhitepaperPlanning a Successful DeploymentDownload now! As an example, Coherence has very few software requirements, but it's important you know that Coherence clusters are certified against Java and not the underlying operating system. Thus, you will need to consider the JVM, monitoring tools and underlying OS when migrating. This free whitepaper explores that fact.Another pitfall that many customers face is the allocation of production resources based on the normal usage requirements of the applications they're looking to support. However, if any hardware failures occur, Coherence will be forced to process more data with fewer resources, thereby straining your systems and affecting customer experiences. Considerations for this event are outlined in both this white paper and the Coherence Administration Guide to help you. These are just a couple of small tastes of the best practices and awareness items explored in this free resource. Coherence utilizes rack safety to safeguard against hardware failure A snippet from the paper: Taking a Coherence Proof of Concept (PoC) application and making it ready for production involves a range of considerations and additional steps, which are not always obvious to architects and developers. The environment of a production application is quite different to that of development and steps to secure, monitor and manage the application will need to be taken. High Availability plans to meet expected SLA’s will need to be put in place, which in turn will involve appropriate procedures to upgrade, patch and recover... Download your free white paper now to continue reading.Don't forget to join us at Oracle Open World for more sessions and technical details!

"How do I take my Coherence installation out of a concept stage and into production?" We often hear questions similar to this from customers, so we're responding with a free whitepaper that explores...

Coherence

Running the Managed Coherence Servers Example in WebLogic Server 12c

With the recent launch of Coherence 12c (12.1.2), I'd like to go into more detail about one of the new features, Managed Coherence Servers, as well as outline how you can run the sample application available with a WebLogic Server 12.1.2 install. Managed Coherence Servers - Introduction Managed Coherence Servers provide tight integration between WebLogic Server and Coherence. This integration allows for a simplified and streamlined development and management environment for distributed applications. The functionality allows end users to build a new archive type (Grid Archive aka GAR), which can be deployed and managed via standard WebLogic Server best practices. Developers can now streamline their build processes to generate GARs. Operations departments can now standardize deployment of Coherence and Coherence applications in the test and production environments.  Traditionally, Coherence has been deployed as a jar incorporated into a Java application (e.g., WAR or standalone Java application) along with a tier of standalone cache server JVMs (often using the DefaultCacheServer class). The embedded java usages (such as in a WAR) are referred to as clients and the standalone cache servers are referred to as servers. The lifecycles of the “clients” and cache servers are managed separately, often manually, with the cache servers managed by custom shell scripts. Application development and deployment in this model can be a complicated process involving many moving parts that require custom management processes. Within the new model, the artifacts required to run Coherence based applications, such as POJO's, server side logic and cache configuration are packaged into a GAR.  This mirrors to a large degree the structure of other Java EE artifacts such as Web Archives (WAR).  This GAR can be deployed and managed in the same using the standard WebLogic Server tools such as: Admin Console WLST  WebLogic Deployer WebLogic Development Maven Plugin  The Coherence cluster's operational configuration and structure is defined within the Admin Console, or via WLST. You can then take advantage of the traditional WebLogic Server constructs such as Clusters (not to be confused with the new Coherence Cluster artifact), to build a tiered application architecture that might include a data tier, application tier, and proxy tier.  The Coherence configuration is targeted to those WebLogic Server Clusters and Managed Servers that are assigned to those WebLogic Server Clusters inherit the operation configuration by being members of those WebLogic Server Clusters.  You can then deploy Coherence based applications, (EARs, GARs) to clusters and they will be automatically deployed to all members of the cluster. Please see the Oracle Coherence YouTube Channel for more in-depth videos and demonstrations of Managed Coherence Servers. Running The Sample Application When you download WebLogic Server 12c and complete the full install, there is a sample application and sample WLST scripts that you can use to get familiar with the new feature. I'll explain the steps to do this below. Downloading WebLogic Server and a supported JDK  Firstly you need to download WebLogic Server 12c from Oracle Technology Network.  You can download the "Generic WebLogic Server and Coherence Installer" or the specific installer for your operating system. If you choose to download the generic installer you must download a certified JDK as well. See here for detailed installation information. In my example I downloaded the latest  "Java SE Development Kit 7u25" available here. Installing WebLogic Server For my example, on Windows 7, I ran a command prompt as administrator, set my JAVA_HOME to my installed JDK and ran the installer. Click Next on the "Welcome Screen", and choose C:\Oracle\Middleware (or appropriate directory) as install location. Click Next to continue. On the next screen ensure you select Complete Install as this will install the examples as well. Select Next to continue. Continue through the installer and after the installation is completed, ensure the "Automatically Launch the Quickstart Configuration Wizard" check-box is checked, and click Finish. When the Config Wizard starts, choose a a password, such as welcome1, and click Create to configure the examples. After the creation is complete, you don't need to start the domain for these examples.  Viewing the Examples Home Page Open the index.html file in the examples home directory. For my install the home directory was c:\Oracle\Middleware\user_projects\applications\wl_server\index.html. This page is the home for all WebLogic Server examples. You can select the Managed Coherence Servers Examples link to view the examples.  There are a number of examples showcasing the new functionality. If your machine doesn't have too much memory, e.g, 4GB or less you can follow the instructions in the Single Server example which will deploy only to a single Admin Server.  Below, I will explain how to run the Multi Server example which uses a tiered approach. Running the Multi-Server Example Open a command prompt and set your WebLogic Server environment by using the setWLSEnv.cmd/ setWLSEnv.sh in your installation. For my install the location of setWLSEnv.cmd is C:\Oracle\Middleware\wlserver\server\bin. Next change to the examples home directory. This is a nice long directory, but in my install the location was C:\Oracle\Middleware\user_projects\wl_server\examples\src\examples\coherence\managed-coherence-servers.  Under this directory there are a number of directories, but as we are going to run the multi-server example, change to the multi-server directory. Next type ant deploy to build and install the application. If ant is not found, ensure you have correctly run setWLSEnv.cmd/sh. After the compilation has completed answer the prompts to enter a password (note this will be displayed in clear text) and hostname for your machine. You can leave the other ports as defaults. The build process will create a new domain called multi_domain with a data tier and app tier and deploy the application to this domain.  On completion, you can then access the URL shown above and run the sample JSP application, shown below.  You can also access the Admin console to view the applications and configuration that was created. Conclusion There are other examples available in the same location showing you how to carry out a rolling redeploy and thread-dump of all managed servers using WLST. See the following links for more information on Coherence 12.1.2 and Managed Coherence Servers.  Oracle Coherence YouTube Channel Developing Coherence Applications for WebLogic Server  Coherence Documentation Home Page Hope you enjoy the new functionality!

With the recent launch of Coherence 12c (12.1.2), I'd like to go into more detail about one of the new features, Managed Coherence Servers, as well as outline how you can run the sample...

Coherence

Cloud Application Foundation 12c Customer Showcase: Orient Overseas Container Line (OOCL)

Continuing our mini-series of customer showcases leading up to the July 31st launch event of Coherence 12c, today we're taking a peek behind the curtains of Orient Overseas Container Line (OOCL). OOCL is a wholly-owned subsidiary of Orient Overseas (International) Limited (OOIL), a public company listed on the Hong Kong Stock Exchange. OOCL is one of the world's largest integrated international container transportation, logistics and terminal companies. As one of Hong Kong's most recognized global brands, OOCL provides customers with fully-integrated logistics and containerized transportation services, with a network that encompasses Asia, Europe, North America and Australasia. OOCL operates a fleet of 700,000 containers with 270 offices in 60 countries and about $6.5 billion in annual revenue. Considered a leader in IT among ocean container carriers, OOCL prides itself on utilizing IT as a competitive advantage -- 1 in 6 OOCL staff members works in IT. It is one of only two ocean carriers profitable in 2011 and had the largest profit in the entire industry for the past five years combined (according to American Shipper magazine). OOCL engaged with Oracle on a multiple-year project to replace core ERP systems. They utilize Oracle WebLogic Suite, Coherence, RAC DB, TopLink, and Oracle Enterprise Manager to address their modern IT challenges. In fact, OOCL has a single WebLogic domain with 30 applications distributed on 75 managed servers with number of applications growing significantly.Join us for the July 31st launch event to hear Matt Rosen of OOCL explain why they're so excited about the new releases of WebLogic and Coherence, specifically in the areas of WebLogic Coherence integration, Coherence GoldenGate Hotcache, Websockets & Maven Plug-in support, and more and what they anticipate as benefits from these brand-new capabilities.

Continuing our mini-series of customer showcases leading up to the July 31st launch event of Coherence 12c, today we're taking a peek behind the curtains of Orient Overseas Container Line (OOCL).OOCL i...

Coherence

Cloud Application Foundation 12c Customer Showcase – Turkcell

Today, I want to highlight Turkcell as one of the key customers contributing to the upcoming Cloud Application Foundation 12c launch event on July 31st. Turkcell is the leading communications and technology Company in Turkey, with 34.9 million subscribers and a leading regional player, with market leadership in five of the nine countries in which it operates with its approximately 69.2 million subscribers. With a reported $1.5 billion in revenue and $10.4 billion in total assets, Turkcell is the only NYSE-listed company in Turkey. I think the two words to define Turkcell's IT environment are "huge" and "complex." Turkcell's main business functions are CRM , Value Added Services (VAS), Billing/Charging and Network operations managing thousands of user sessions and millions of business transaction every day with 4000+ operating systems, 150+ Oracle databases in production, and 1700+ Oracle WebLogic Server instances.Weblogic Server is the key product in Turkcell's middleware strategy. Mainly used in CRM, VAS and Billing business lines (where Java EE applications are widely used) their middleware platform predominantly uses Oracle Weblogic Server. Key reasons Turkcell prefers Weblogic server are as follows: Java EE applications perform better on Weblogic Server under high load of transactions/li> High availability and load balancing solutions like application and server clustering and failover , multi data sources are key features Turkcell uses to increase service availability Turkcell’s business needs changes and grows frequently as their infrastructure must. Weblogic Server adapts to business requirements with ease of scalability and flexibility. Operational excellence – Turkcell believes in simplicity, knowledge convergence and productivity in daily operations. Monitoring capabilities over WebLogic management beans are far stronger and functional compared to others. Recently, Turkcell has adopted in-memory computing with Oracle Coherence. Their two main goals with taking their high-transactional data in-memory from physical databases or other service backend systems are: To increase performance of our core applications and achieve more transactions per second Off load backend data Turkcell has been using Oracle Coherence largely for caching frequently accessed data in many mission critical business functions including, call center, order management, campaign management and vast services like SMS processing and so on.Oracle Weblogic Server and coherence infrastructure today serves majority of Turkcell customers and partners, from every subscriber to internal staff. Join us for the July 31st launch event to hear why Turkcell is excited about the new releases of WebLogic and Coherence, and what they anticipate as benefits from these brand-new capabilities.

Today, I want to highlight Turkcell as one of the key customers contributing to the upcoming Cloud Application Foundation 12c launch event on July 31st. Turkcell is the leading communications and...

Coherence

Fresh Resource: Solving Data Management and Scalability Challenges with Coherence

Hot off the presses, we have a new, highly-informative whitepaper for you!Today's mobile, cloud and social demands are increasingly putting your infrastructure in a vice-grip. If you throw in the growth of the "internet of things," you get even more nervous about how you're going to handle the onslaught of data and processing needs. Sound familiar? Oracle Coherence 12c allows you to cache and process data in real time. In this whitepaper, you'll discover how Oracle Coherence 12c is helping you meet the challenges you face, including: caching and processing data in real-time offloading back-end infrastructure and shared services managing applications with native, intuitive cloud management much more! An excerpt: ...For many applications, the pertinent data resides in the database or a backend operational data store, often on a mainframe computer that was never intended to handle the increased demand it is now tasked with. One of the leading problems is shared services overloading business infrastructures, creating latency issues, unpredictable outages and poor customer experiences. Further, these back-end services cannot scale easily and cost effectively as the business grows and buying patterns change. All of these issues can lead to a poor user experience and subsequent loss of business...   > DOWNLOAD THIS FREE RESOURCE NOW Also, don't forget to join us for our Coherence 12c launch event later this month:July 31st10am, Pacific TimeRegister

Hot off the presses, we have a new, highly-informative whitepaper for you!Today's mobile, cloud and social demands are increasingly putting your infrastructure in a vice-grip. If you throw in the...

Coherence

Introducing Oracle Coherence 12c

Today Oracle announced the generalavailability of Coherence 12c. This release has been a couple years in themaking, and contains many features that allow you to deliver next-generationapplications while dynamically scaling your applications to meet increasedcloud and mobile demands. Hereis a little teaser of some of the new functionality you can expect from therelease. For more information, please register for ourlaunch event where you will learn howCoherence 12c will help you: Power applications with always-accurate, real-timedata Eliminate "stale data" problems Scale serversdynamically Adapt tounexpected user spikes automatically Obtain speed andoperational advantages through Cloud Application Foundation integration Simplifyoperations with native cloud management Enhance yourdeployment options for high availability Much more! Coherence 12c ensures yourdata is up to date in real-time, and integrates seamlessly with Oracle WebLogicServer 12c to deliver dynamicapplications. Overall, Coherence 12c is designed to deliver efficiencies in both performance andmanagement, while allowing you to scale your applications to meet increasingmobile and cloud demands on your infrastructure. Download Oracle Coherence 12c today. Join our online launch event! Cloud Application Foundation Launch July31st 10am,Pacific Time REGISTER NOW

Today Oracle announced the general availability of Coherence 12c. This release has been a couple years in the making, and contains many features that allow you to deliver next-generationapplications...

Coherence

Can't Miss Event: Oracle Coherence 12c Launch Webcast

We're super-excited around here about the impending launch of Oracle Coherence 12c as part of the Cloud Application Foundation launch this month! We want you to join us for the Cloud Application Foundation launch event to learn more about Coherence's ability to deliver applications with a mission-critical cloud platform, enhance deployment options for high availability and simplify operations with integrated products and management. Scale your applications to meet mobile and cloud demands! Oracle Cloud Application Foundation Launch Including Oracle WebLogic Server, Oracle Coherence, Oracle Enterprise Manager and Oracle Development ToolsJuly 31st, 2013 10am Pacific Time >> Register now! (of course, it's free) This will be the first release of Coherence we're making available at the same time as an Oracle WebLogic Server release - and that's not a coincidence. One of the main focus areas of this launch is the operational simplicity that we want you to enjoy, and that includes a tight integration not only with WebLogic Server itself, but also with cloud management tools (Enterprise Manager) and developer technologies - like JDeveloper, Eclipse tools, ADF Mobile and more - to ensure you can be productive out of the box on day one. The word is, there are even some heavy-duty capabilities Coherence will be delivering around real-time data processing, elastic scalability, developer technology friendliness and even some deep integration with Oracle Database 12c, which is launching on July 10th. But, we're already giving away too much. We look forward to seeing you there!

We're super-excited around here about the impending launch of Oracle Coherence 12c as part of the Cloud Application Foundation launch this month! We want you to join us for the Cloud...

Coherence Special Interest Group

Bay Area Coherence SIG on May 1

Come meet Coherenceengineers, product managers, partners and customers at the BayArea Coherence Special Interest Group (CSIG), Wednesday, May 1st, 5:30pm PT at the Oracle Conference Center, 350 Oracle Parkway, Room 203, RedwoodShores, CA. Topics include: Coherence Roadmap and Community Update (Craig Blitz, Senior Group Product Manager, Oracle) In this talk, Craig will discuss the next Coherence release, including the new GoldenGate HotCache feature for synchronizing with databases and Managed Coherence Servers for full cluster and container management. He will also discuss the new community development process that extends the popular Incubator model. Finally, we will discuss some projects in the labs slated to be delivered in upcoming releases. Oracle NoSQL Database and Coherence Integration - A Solution For Big Data Problems (Dave Rubin, Director, Oracle NoSQL Database Development) Dave will present a technical overview of Oracle NoSQL Database, a massively scalable, transactional key-value store. Also presented will be big data use cases that provide a rationale for an integrated solution and explain why such an integrated solution is key to meeting the stringent latency and scale requirements in these scenarios. Session Management with Oracle Coherence- Coherence*Web Usage Patterns and Features (Torkel Dominique – Principal Member of Technical Staff, Oracle) Coherence*Web is being used by companies looking for the scalability, operational agility, and availability benefits that come with managing sessions in a separate tier. This talk will present how Coherence*Web is being used today, and will then dive into some of the more powerful features of Coherence*Web, including session models and locking options.  Be sure to keep up with theCSIGs in your area on our wiki.

Come meet Coherence engineers, product managers, partners and customers at the Bay Area Coherence Special Interest Group (CSIG), Wednesday, May 1st, 5:30pm PT at the Oracle Conference Center, 350...

Coherence

Thomson Reuters Oracle Customer Showcase, April 15-16, Eagan, MN

Don’t miss this opportunity to spend a day hearing abouthow Thomson Reuters uses Oracle technology to transform their business. The ThomsonReuters Oracle Customer Showcase is an interactive event that will be heldApril 15-16 at Thomson Reuter’s office in Eagan, Minnesota. The showcase featuresa variety of presentations from Thomson Reuters, Oracle Executives and line ofbusiness experts, including how Oracle Coherence and Oracle Database poweredThomson Reuter’s WestLaw project (described in more detail below). There willbe tours of the Thomson Reuters data center, a hands-on solution pavilion,breakout sessions and networking opportunities. And to top it all off,attendees will be treated to a Minnesota Twins Baseball game. ThomsonReuters Westlaw Improves Scalability and Performance with Oracle Coherence Thomson Reuters Westlaw is one of the primary onlinelegal research services for lawyer and legal professionals, providingproprietary data services and information from more than 30,000 databases ofcase law, statutes, synopses, treatises, best practices, news articles andpublic records. A million times a day, Thomson Reuters Westlaw provideslawyers with the answers they need about historical law and the latest cases froma day, week or month ago so they can address their clients’ legal issuesquickly and accurately. With the nature of search changing almost daily, ThomsonReuters Westlaw decided to optimize their infrastructure around Web 2.0 todeliver cutting edge performance and capabilities. Their first challenge was tocreate an advanced search algorithm that would identify relevant documentsusing term matching, legal concepts, document characteristics and searchpatterns, so lawyers could do research faster with confidence they were notmissing anything. Next, the team created a sleeker interface and otherfeatures to improve workflow and help customer keep better track of theinformation they found. Then, they had to find enabling software and hardware thatwould scale quickly, be available 24/7 and that was flexible enough to evolve andscale to meet constantly changing client needs. After meeting with many vendorsand evaluating their offerings, Thomson Reuters Westlaw selected Oracletechnology. Oracle Coherence and Oracle Streams addressed Westlaw’sdata consistency needs enabling them to maintain high availability despitebeing more computationally demanding than previous versions of the application.This was made possible because Oracle Coherence and Oracle Streams share the cacheacross all nodes so, if a particular node fails, another node picks up wherethe other left off. Oracle Coherence and Oracle Streams also provide dataconsistency across functions i.e. a lawyer wants to search terms highlighted onboth onscreen and printed results. Oracle Coherence also addressed Westlaw’s need to scaleto an unprecedented degree immediately. Coherence provided the ideal solutionfor scalable data management. Scaling both capacity and throughput was easilyachieved by adding more nodes to the Coherence cluster. Find out more about how the improved performance, scalabilityand availability Oracle Coherence provides helps Thomson Reuters Westlaw betterserve their clients and create unique competitive advantage. For MoreInformation Register for: ThomsonReuters Oracle Customer Showcase Read the article: NextDiscovery: Thomson Reuters, Legal, sets new precedent for search performance Visit the Homepage: Coherence Learn more: ThomsonReuters Westlaw line-height: 115%; font-family: Arial, sans-serif;">Stayconnected Follow Coherence: Stay Connected... Coherence: Cloud Application Foundation (CAF):

Don’t miss this opportunity to spend a day hearing about how Thomson Reuters uses Oracle technology to transform their business. The Thomson Reuters Oracle Customer Showcase is an interactive event...

OpenWorld & JavaOne

Be Part of the Conversation: Plan to present at Oracle Open World or JavaOne 2013

Have you ever attended conferencesessions and thought, “Gee, I know more about this topic—I could be thepresenter!” Or, have you ever thought, “This conference is missing some topicsI’d like to hear more about. Why don’t they have a session on “x”?” Now is your chance to change that. The Call for Proposals for Oracle OpenWorld and JavaOne,held September22 - 26, 2013 at the Moscone Center in San Francisco, is open until April 12. Wewant to hear your stories. You are the oneliving these topics and the ones who have to apply the ideas back home in reallife. Don’tmiss this opportunity to share your thoughts, innovative ideas and experienceswith your peers and other technology experts. Hereare some presentations from JavaOne 2012 to help get you inspired. Usingthe new javax.cache Caching Standard. Hear Greg Luck and Carmen Purdy’s describehow to: • Abstractyour caching implementation • Use therich and modern API • Use thenew caching annotations from Java EE • Usejavax.cache with existing Java EE 6 and Java SE applications • Plug invarious implementations DistributedCaching to Data Grids: The Past, Present, and Future of Scalable Java Hear about the history of Javadistributed caching and its most important innovations. Learn how thistechnology has become integral to enterprises building reliable and scalablelarge-scale distributed systems (including financial services, travel, onlinegaming, retail, and many others). Get a glimpse into how the technology mightevolve and how enterprises will adopt it. NoSQLUsage Patterns in Java Enterprise Applications Chris Davis, Nike EngineeringDirector and Randy Stafford, Oracle Architect At-Large explore and codify numberoususage patterns that have emerged for NoSQL technologies in the context ofmodern enterprise application architectures, illustrated by examples drawn from launched productionapplications. They present a model and a vocabulary for describing various usesof NoSQL stores in Java EE applications and describe the consequences, bothpositive and negative, of each usage pattern. ShardingMiddleware to Achieve Elasticity and HIgh Availability in the Cloud Cameron Purdy, VP Development, Oracleand Brian Oliver, Senior Solutions Architect, Oracle,examine some of the fundamental and somewhat radical departures in systemdesign when "sharding paradigm" is applied totraditional middleware. See how sharding is influencing the way enterpriseproducts such as GlassFish, Oracle WebLogic, and Oracle Service Bus aredesigned. WhyThere’s No Future in Java Futures Hear Brian Oliver, Senior PrincipalSolutions Architect and Mark Falco, Consulting Member Technical Staff, examinehow Java Future affects the design and implementation of highly concurrentmultithreaded systems. Find out how, although being designed to facilitateconcurrent programming, they often have the opposite effect, unnecessarilycausing developers to burn threads and actually encouraging the development ofblocking synchronous systems. Learn about several simple alternatives,including the notion of Deferreds and Collectors, together with a DSL-styleframework for testing such implementations. Submit your proposal today! OracleOpenWorld 2013 Call for Proposals JavaOne2013 Call for Papers

Have you ever attended conference sessions and thought, “Gee, I know more about this topic—I could be the presenter!” Or, have you ever thought, “This conference is missing some topicsI’d like to...

Coherence

Oracle Coherence Near Cache Best Practices

"There are only two hard things in Computer Science: cache invalidation and naming things." - Phil Karlton A typical Coherence application includes a tier of cache servers (JVMs dedicated to storing and managing data) and a tier of cache clients (JVMs that produce/consume data). This tiered approach is especially common with large web sites that use Coherence to manage sessions and/or content. With this type of architecture, caches that are accessed often can be configured as near caches. A near cache is a local in-memory copy of data that is stored/managed in cache servers. Coherence supports near caching in all native clients (Java, .NET, and C++).  The obvious advantage of a near cache is reduced latency for accessing entries that are commonly requested. Additionally, a near cache will reduce overall network traffic in several ways. Requests for cache entries that exist in the front of the near cache will be serviced by the client making the request, thus eliminating the need for a request to the cache server. If an entry does not exist in the front of the near cache, a single request will be made to retrieve that entry from the cache server, even if multiple threads on the client are requesting the same key. Another benefit of a near cache is that entries are stored in object form, meaning that clients don't have to spend extra CPU cycles on deserialization.  The aggregate of these benefits is increased scalability of a system. In order to determine if a near cache is appropriate, it is important to understand how near caching works internally. This knowledge is required in order to correctly configure near caches. Invalidation is the biggest challenge for near caching. Storing entries in a client is easy, but detecting when changes are made to those entries in other clients is more difficult. Coherence provides several different strategies for near cache invalidation. Listener based With the listener based strategies, Coherence will register map listeners that receive events when the contents of the "back" cache are modified. When these events are received by the client, the affected entries will be removed from the front of the near cache. This ensures that subsequent requests will force a read-through to the cache server in order to fetch the latest version. Today Coherence supports two listener based strategies: PRESENT and ALL. Expiry based This strategy does not use map listeners to invalidate data, which means there is no way for the front of the near cache to know when entires are updated. This requires the configuration of an expiry on the front of the near cache. When an entry expires, this will force a read-through to the cache server to fetch the latest copy. If an expiry-based strategy is used, the invalidation strategy should be set to NONE. The following is a description of each of the strategies in detail:  PRESENT This invalidation strategy indicates that each near cache will register a listener for every entry present in the front of the near cache. Pros: The near cache will only receive events for entries that it contains, thus greatly reducing the amount of network traffic generated by the cache servers. Cons: This strategy results in increased latency for cache misses. This is because a map listener must be registered for a key before the entry is retrieved from the cache server. Since listener registrations are backed up (just like cache entries) the message flow looks like this: The increased latency for the listener request will vary depending on the network. A good rule of thumb would be ~1ms of overhead for every cache miss. Another side effect is slightly higher memory consumption on the cache servers to maintain the key registrations. The exact overhead depends on the size of the keys. This is an area that we plan on optimizing in a future release. This strategy works best for near caches that: Have access patterns where certain JVMs will access a small portion of the cache - for example, web applications that use sticky load balancing will mostly access the same subset of sessions stored in a cache Have back caches that won't expire or evict data often since there is a latency penalty for cache misses  ALL The invalidation strategy of ALL indicates that each near cache will register a single map listener to receive events on all updates performed in the cache. This includes events for entries not present in the near cache. Pros: This strategy maintains the coherency of the near cache without the extra latency associated with cache misses. Cons: Every time an entry is updated on the storage tier, an event will be delivered to each near cache. For clusters with a large client tier, this can generate a large amount of network traffic. This strategy can be especially problematic when bulk updates are preformed via: Bulk loading/seeding of the cache  (i.e. populating the cache from a database) Clearing the cache Cache server failures which cause redistribution which cause mass evictions due to exceeding of high units This strategy works best for near caches that: Contain a small amount of data (low hundreds of megabytes or less) Have access patterns that guarantee that a significant portion of the cache will be accessed by each client When using the ALL strategy, it is important to avoid bulk updates or deletes in order to limit the number of map events that are generated. NONEThe NONE invalidation strategy does not use map listeners; therefore entries in the near cache are never invalidated. Since there is no invalidation, the front of the near cache must be configured with an expiry which will force entries to be removed and a read-through to the back tier. The front of the near cache must be configured with an expiry. Here is an example: <near-scheme>  <scheme-name>near</scheme-name>  <front-scheme>    <local-scheme>       <!-- front expiry -->      <expiry-delay>1m</expiry-delay>     </local-scheme>  </front-scheme>  <back-scheme>    <distributed-scheme>      <scheme-ref>partitioned</scheme-ref>    </distributed-scheme>  </back-scheme>  <invalidation-strategy>none</invalidation-strategy></near-scheme> <distributed-scheme>  <scheme-name>partitioned</scheme-name>  <backing-map-scheme>    <local-scheme>      <expiry-delay>5m</expiry-delay> <!-- back expiry -->    </local-scheme>  </backing-map-scheme>  <autostart>true</autostart></distributed-scheme> An expiry in the backing map is not required; it is shown here for illustrative purposes in order to distinguish between front and back expiry. Pros:This strategy is the most scalable since it does not require delivering map events for every update made in the cache. It does not have to pay a latency penalty for misses. Furthermore, this strategy works best for caches that require a high rate of updates. Cons:NONE near caches will require some tolerance for stale data. Note however that an expiry of as little as a few seconds can make this strategy a good compromise between low latency access and scalable performance for the cluster. This strategy works best for near caches that Have back caches containing large amounts (tens of gigabytes) of data Have large client tiers (many dozens or hundreds of JVMS) Have a requirement for bulk updates  AUTO If no invalidation strategy is selected, the AUTO strategy is the default. As of Coherence 3.7.1, this defaults to the ALL strategy. This is subject to change in future releases of Coherence; in fact this will be changing to PRESENT in the next release. Therefore, it is advised to always explicitly select an appropriate invalidation strategy for every near cache deployment. Note that applications that are write-heavy may be better off without a near cache. This especially applies for the ALL and PRESENT strategies where every update to the cache will cause the propagation of map events. In order to determine near cache effectiveness, look at the HitProbability attribute in the near cache MBean. This MBean is of type "Cache" and the ObjectName will contain "tier=front" which indicates that it is a near cache. As a rule of thumb, ensure that near caches are yielding a hit probability of at least 80%. Anything less may merit the use of NONE or the complete removal of the near cache. Other Considerations In addition to selecting an appropriate invalidation strategy, there are a few other considerations to be made when using near caches. The first has to do with mutable values placed into a cache. Consider the following sequence in a single client: Thread 1: value = cache.get(key)value.setX(...)cache.put(key, value) Thread 2: value = cache.get(key)value.setY(...)cache.put(key, value) When this is performed against a distributed/partitioned cache, each thread gets its own deserialized copy of the cached item. Therefore Thread 2 won't see the modification made to property value X by Thread 1. However if this operation happens on a near cache, it is possible for threads to see mutations made by other threads to a cached value. Note that this is no different than using a ConcurrentHashMap or any other thread safe Map implementation in a single JVM. The major difference is the client behavior with a partitioned cache vs a near cache. The recommended best practice is to use immutable value objects when using a near cache. If existing cache objects must be mutated, consider using an entry processor to modify the value.  The second concern has to do with cache configuration when using a proxy. Consider the following sample configuration: <near-scheme>  <scheme-name>example-distributed</scheme-name>  <front-scheme>    <local-scheme>      <high-units>10000</high-units>    </local-scheme>  </front-scheme>  <back-scheme>    <distributed-scheme>      <scheme-ref>example-distributed</scheme-ref>    </distributed-scheme>  </back-scheme>  <invalidation-strategy>PRESENT</invalidation-strategy></near-scheme> When a cache server (storage enabled member) reads this configuration, it will skip the creation of the near cache (since this is not a client) and instead will create the back-scheme as defined by the configuration. When a storage disabled member reads this configuration it will do the opposite - it will create the front of the near cache and skip the creation of the back-scheme. This presents an interesting scenario when proxy members are added into the cluster. The best practice for proxy members is to disable storage in order to preserve heap for handling client requests. This means that the proxy will create the front of the near cache, just like any other storage disabled member. This is not desirable because: The proxy will consume more memory and CPU for little benefit. The main function of the proxy is to forward binary entries to Extend clients. Therefore, the (deserialized) contents of the near cache are not being consumed in the proxy. This not only leads to more memory consumption, but also more CPU cycles for deserializing cache entries.  If the proxy is serving more than one client, it is likely that the near cache will experience a lot of turnover, which results in more overhead for GC.  The cache configuration for a proxy member should never contain a near-scheme. Conclusion Near caching is a powerful tool of the Coherence in-memory data grid that - when used judiciously - can improve latency, reduce network traffic, and increase scalability for a Coherence application. The decision to use a near cache should always be made with the invalidation strategy as an explicit consideration. 

"There are only two hard things in Computer Science: cache invalidation and naming things." - Phil Karlton A typical Coherence application includes a tier of cache servers (JVMs dedicated to storing...