<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
   <title>Event Driven Architectures</title>
   <link rel="alternate" type="text/html" href="http://blogs.oracle.com/pwalker/" />
   <link rel="self" type="application/atom+xml" href="http://blogs.oracle.com/pwalker/xml/rss.xml" />
   <id>tag:blogs.oracle.com,2008:/pwalker//274</id>
   <updated>2008-11-13T08:14:24Z</updated>
   
   <generator uri="http://www.sixapart.com/movabletype/">Movable Type Enterprise 1.52-en-voltron-r47459-20070213</generator>

<entry>
   <title>Event Processing in Enterprise Operations</title>
   <link rel="alternate" type="text/html" href="http://blogs.oracle.com/pwalker/2008/11/event_processing_in_enterprise_operations.html" />
   <id>tag:blogs.oracle.com,2008:/pwalker//274.8516</id>
   
   <published>2008-11-10T18:46:45Z</published>
   <updated>2008-11-13T08:14:24Z</updated>
   
   <summary>The article &quot;Returns on Event Processing&quot; focuses on identifying and capturing returns within an enterprise. Paradoxically, event processing (EP) systems can&apos;t perform those functions by themselves, but rather EP sits in the center of the value identify-capture process. Let&apos;s define...</summary>
   <author>
      <name>perren.walker</name>
      <uri>http://blogs.oracle.com/pwalker</uri>
   </author>
         <category term="Event Driven Architectures" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en-us" xml:base="http://blogs.oracle.com/pwalker/">
      <![CDATA[<p>The article "Returns on Event Processing" focuses on identifying and capturing returns within an enterprise. Paradoxically, event processing (EP) systems can't perform those functions by themselves, but rather EP sits in the center of the value identify-capture process. Let's define value-capture as minimizing or avoiding a cost incurrence or capturing a profitable sale. In addition, there is both a first-mile and a last-mile in a successful event-processing implementation and Business Intelligence (BI), Business Activity Monitoring (BAM), and Business Process Management (BPM) play important roles.  This article examines event processing in enterprise operations and how BI, BAM, and BPM complement event processing in achieving returns.<br />
 <br />
<strong> Event Processing and Business Intelligence (BI)</strong></p>

<p> The event processor <em>identifies</em> value among the events it is processing; the event processor does not <em>determine</em> value. The intrinsic determination of value is made within your organization or perhaps by a marketplace outside your organization. Business intelligence analytics functions in first-mile in event processing by unearthing opportunities in supply chain, Financial, customer, HR workforce, or your IT/Organization. It is <em>your</em> determination of value, translated into Expression Query Language (EQL), Continuous Query Language (CQL) that enables opportunity identification. The event processor continuously looks for those opportunities and when an identification is made, then the last-mile of value-capture can take place. </p>

<p> <strong>Event Processing and Business Process Management (BPM)</strong></p>

<p> Business Process Management (BPM) is one example of "last-mile value-capture." Opportunities identified by the event processor can kick off a human centric workflow involving decisions and action that results in value-capture. For example, your event processor might identify that a weather pattern is going to delay the arrival of material to a key customer. Both an automated alert can be sent to the account representative, in addition to starting a human process to assess and escalate problem to upper management. A decision can be make to re-route shipment or manage the customer's expectations.</p>

<p><strong> Event Processing and Business Activity Monitoring (BAM)</strong></p>

<p> Business Activity Monitoring (BAM) is also a component of "last-mile value-capture," as BAM processes and presents data as it occurs within the enterprise. There is value in visualizing and analyzing the information but it is the decisions derived from BAM analysis that result in financial impact. For example in the last week of a financial quarter, a competitor's price drop may take away sales resulting in top line miss for your organization. BAM's ability to present declining revenue, as it is occurring, is critical to identifying and quickly responding to the trend in order to make your numbers. Business Intelligence reports generated daily or weekly may not be timely enough to make the necessary business adjustments.</p>

<p><strong> Event Processing as an Operational Tool</strong></p>

<p> Event processing sits in the center of the process improvement analyze, act, measure life cycle. Event processing takes on an operational role looking for both large and small opportunities on a 24/7 basis. Returns on event processing are achieved when the event processor "acts" upon an identified opportunity.</p>

<p> Furthermore BI, BAM, and BPM are complementary technologies. Business Intelligence (BI) and Business Activity Monitoring (BAM) play important roles analyzing, measuring, and reporting. Event Processing (EP) and Business Process Management (BPM) complement each other in their ability to act. That action will hopefully improve business results that are then analyzed, measured, and reported. In short, event processing is a very powerful operational tool that can directly impact customer satisfaction, margin and revenues.<br />
</p>]]>
      
   </content>
</entry>
<entry>
   <title>Returns on Event Processing</title>
   <link rel="alternate" type="text/html" href="http://blogs.oracle.com/pwalker/2008/09/returns_on_event_processing.html" />
   <id>tag:blogs.oracle.com,2008:/pwalker//274.6823</id>
   
   <published>2008-09-19T17:17:25Z</published>
   <updated>2008-09-22T23:13:24Z</updated>
   
   <summary> Returns on event processing are highly specific to the industries in which they operate. Event processing does not lend it self to general return on investment characterizations of enterprise technologies. For example, middleware technology can enable customer self-service and...</summary>
   <author>
      <name>perren.walker</name>
      <uri>http://blogs.oracle.com/pwalker</uri>
   </author>
         <category term="Event Driven Architectures" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en-us" xml:base="http://blogs.oracle.com/pwalker/">
      <![CDATA[<p>          Returns on event processing are highly specific to the industries in which they operate.  Event processing does not lend it self to general return on investment characterizations of enterprise technologies.  For example, middleware technology can enable customer self-service and consequently reduce the capacity requirements of a human sales and service organization.  Organizations also employ middleware technology for internal facing self-service applications.   Another example, data backup and disaster recovery can be viewed as a form of self-insurance.  Both middleware and data backup have broad business value propositions that cross industry boundaries, with investment returns clearly tied to use cases.  </p>

<p>	For organization that has not evaluated the business impact of event processing, how can one imagine the potential returns on event processing?  Identifying event processing opportunities within your organization can be extremely profitable resulting in improved processes, customer interaction and ultimately margin expansion.  At worst, asking and exploring this question may lead to a better understanding of your internal processes and value flow within your organization.  Lets take a look at existing event processing use cases and examine how event processing is used to identify and capture value:  </p>

<p><BLOCKQUOTE><br />
Aircraft collision avoidance systems, their investment return is immeasurable preventing the loss of life.  Yet in the shipping industry, some 18 years after the Exxon Valdez, there are still maritime collisions occurring such as the Cosco Busan’s collision with the San Francisco Bay Bridge on November 7, 2007.  So clearly, very large opportunities still exist for event processing today.<br />
</BLOCKQUOTE></p>

<p><BLOCKQUOTE><br />
With algorithmic trading, event processing enables the profitable capture of very small returns.  Achieving a one-tenth-of-one-percent net-return (0.10%) on selling a security might not sound very appealing; however, if you were able to achieve that net-return every day without loss, over 200 trading days you would have a 22.12% compounded-return.  <br />
</BLOCKQUOTE></p>

<p>The above two investment returns could not seem more different - identifying and capturing lots very small returns and identifying and avoiding, relatively infrequent, costly outcomes. Both examples are enabled by the low marginal cost of event processing.  In both cases significant returns are achieved.  When evaluating event processing opportunities within your enterprise, keep these examples in mind as you look to identify and capture value that may be left on the table with your existing processes.  There is almost no return too small for an event processor to programmatically detect and capture. </p>

<p>Now lets examine the role that human interaction versus automated processing plays event processing opportunities.  Take for example programmed trading, if a true arbitrage opportunity exists, frequently trades are automatically placed - no human intervention is required.  If the opportunity is less certain with a larger time-window, a trader may be asked to make the final decision.  With human intervention there is a higher cost associated with human involvement and hopefully a corresponding return that justifies that cost.  The net effect is business process segmentation, where a potential return can be matched to a process resulting in a profitable transition.  This finance example is an internal process, however, there is no reason why customer-facing process couldn’t benefit as well.</p>

<p>Both examples outlined above are core to their respective business and the total return from each respective use case can be immense.  With the dramatic fall in the cost of event processing development and maintenance, it is worth exploring the profitability of event processing within every area your organization, not just the core business.  <br />
</p>]]>
      
   </content>
</entry>
<entry>
   <title>Event Servers, A Disruptive Technology</title>
   <link rel="alternate" type="text/html" href="http://blogs.oracle.com/pwalker/2008/09/event_servers_a_disruptive_tec_1.html" />
   <id>tag:blogs.oracle.com,2008:/pwalker//274.6126</id>
   
   <published>2008-09-02T16:41:46Z</published>
   <updated>2008-09-04T05:09:57Z</updated>
   
   <summary>Event processing has been around for quite some time - programmed trading, systems management software, and windowing systems are just a few examples of specialized event processing. Virtually every program maintains state and executes logic against changes in data state....</summary>
   <author>
      <name>perren.walker</name>
      <uri>http://blogs.oracle.com/pwalker</uri>
   </author>
         <category term="Event Driven Architectures" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en-us" xml:base="http://blogs.oracle.com/pwalker/">
      <![CDATA[<p>Event processing has been around for quite some time - programmed trading, systems management software, and windowing systems are just a few examples of specialized event processing.  Virtually every program maintains state and executes logic against changes in data state.  So why there is such an industry buzz around event processing and what impact can an event server have on your enterprise? </p>

<p>For the first time event processing is being performed by off the shelf logic components that are currently converging toward a industry standard.  This is in contrast to event processing traditionally being performed in custom logic.  The paradigm shift in event processing is similar to what took place with databases.  Data was housed in flat files and proprietary APIs accessed and manipulated that data.  In the broadest of strokes, relational data theory in combination with indexes, transactions and the Structured Query Language revolutionized data storage and retrieval. Just as with databases, it is a combination of technologies that enables event processing’s disruptive effect. </p>

<p>What elements make event processing a disruptive technology? </p>

<p>    * Pluggable container<br />
    * Event Processing Network<br />
    * Expression Query Language / Continuous Query Language<br />
             o filter, correlate, aggregate, and merge data<br />
             o language level time based processing<br />
    * Local and distributed cache<br />
    * Low latency processing<br />
 <br />
<strong><br />
Pluggable container:</strong</p>

<p>The container allows for the aggregation of multiple data streams into a single memory space.  Adapters translate heterogeneous protocol and data specific formats into plain-old-java-objects (POJOs) representing events that then flow through the event server.  Stream based data fusion can now take place with data that previously could not be interrelated.  The complex event processor can take these multiple data streams, aggregate them, filter, correlate, split and merge them.  This single memory space is a major component of low-latency event processing – eliminating sources of latency in order to obtain a results in the shortest time possible.   </p>

<p>Furthermore, a pluggable container allows for multiple types of event processing, not just the window-based processing of a complex event processor.  A rules engine or custom logic in combination with more complicated structures such as Markov chains, Bayesian networks or machine learning algorithms are easily integrated into the container.</p>

<p><strong>Event Processing Network:</strong></p>

<p>The event processing network (EPN) enables various types of event processing within the same process space.  An EPN is composed of nodes connected by streams.  Events represented as POJOs can flow from node to node.  At each node you can have separate type of processing (continuous query, rules based, etc).  This modular approach can result an application that is more easily debugged and maintained. Each node of the network can be separately unit tested and performance tuned. Furthermore, the event processing network provides the granularity to bind various threading models to each node.   </p>

<p><strong>Expression Query Language / Continuous Query Language</strong></p>

<p>EQL and CQL lie at the heart of modern event processing.  These languages are Structured Query Language (SQL) like, however, instead of operating on tables, they operate on streams of POJOs.  If you were to write an event processing from scratch, your application would implement the features (aggregation, filtering, correlation, etc.)  found in these languages. EQL and CQL provide the following capabilities at a language level:</p>

<p>          o <strong>Aggregation</strong><br />
                + As mentioned above, the container aggregates data within a process space, but from there the query engine performs a second level of aggregation where multiple streams can be inputted into a single event processing window. It is within these windows that events of differing origins and types can be filtered, correlated, split and merged.  </p>

<p>          o <strong>Filtering</strong><br />
                + Just as it is important to determine which events are relevant, removing “noise” in data streams allows for more efficient data processing down stream.  Just as SQL returns results from a table that meet the criteria from the WHERE clause, EQL and CQL output events that meet the WHERE clause of the statements that they are executing.</p>

<p>          o <strong>Correlation</strong><br />
                + Correlation can take place in the joining of two separate event streams based on shared attributes, or pattern matching can be applied to windows of events.  Event correlation is the process of establishing a relationship between or among events.  </p>

<p>          o <strong>Merging and splitting data</strong><br />
                + Merging two data streams can acting as form of pre-processing, simplifying down-stream-processing .  Only related data between the two streams will be forwarded on.  For example, you may want the most recent readings (events) from two sensors, but that information is not valuable if the readings are more than one second apart.  You can “merge” two streams of sensor data, creating a single output stream of objects that meet your time-based requirements.</p>

<p>          o<strong> Language level time based processing</strong><br />
                + EQL/CQL languages have temporal qualities that allow them to establish time based windows, freeing the developer from managing time based operations within their own code.</p>

<p>          o <strong>Local and distributed cache</strong><br />
                + Caching services can be used to share data among processing nodes.  Or, when directly referenced in CQL/EQL queries, caching can aggregate less frequently changing data with continuously streaming data, thus reducing processing and memory resources.  When placing data into a Coherence distributed-cache, data will automatically benefit from Coherence’s highly-available architecture.  </p>

<p><strong>Low latency processing</strong></p>

<p>    Many applications require low-latency processing.  For example in algorithmic trading, if a trading decision takes longer to calculate and execute than market movement, the data is stale and the trading decision is no longer relevant.   </p>

<p>    JRockit Realtime, is the anchoring point in low latency processing.  JVM pause times due to garbage collections (GC) are minimized by the deterministic GC algorithm.  Furthermore, JRockit provides facilities for run-time latency analysis.  You can record observations of the application as it is running, examine application execution for latency bottlenecks and tune your application code accordingly.  Lastly, if your architecture includes distributed cache, Coherence’s cache latency can drop when running on top of JRockit Realtime.<br />
    JRockit Realtime enables Oracle's Event Server to meet the most demanding low latency event processing use cases. </p>

<p><br />
As a disruptive technology, what existing technologies are Event Servers displacing?</p>

<p>Event servers dramatically reduce the cost of event processing and disrupts custom coded event management. </p>

<p>   1. Use cases that previously could not justify the cost of custom event processing software development and maintenance can now achieve a return on investment. <br />
   2. Existing event applications can benefit from migration to event servers, freeing resources working on event infrastructure to focus on business event logic - ultimately reducing application costs.<br />
   3. Batch processing of data resulting in exception reports can be translated into continuous event processing algorithms.  Exceptions can be delivered within milliseconds of their occurrence.  For a six sigma process this may be critical in monitoring, identifying and immediately responding to defect rates.<br />
   4. Event processing technology is now affordable at a departmental level.  Event processing can be centrally managed or brought to any operational location of an enterprise.<br />
   5. As a processor of sensor based data, event servers bring down the overall cost of a sensor based solution.</p>

<p></p>

<p></p>

<p></p>

<p></p>

<p> </p>]]>
      
   </content>
</entry>
<entry>
   <title>Object Behavior Within Oracle’s Complex Event Processing Server</title>
   <link rel="alternate" type="text/html" href="http://blogs.oracle.com/pwalker/2008/08/object_behavior_within_oracles.html" />
   <id>tag:blogs.oracle.com,2008:/pwalker//274.6053</id>
   
   <published>2008-08-26T19:01:05Z</published>
   <updated>2008-08-30T09:01:21Z</updated>
   
   <summary>This article highlights object bundling behavior within the Complex Event Processing Network. Object bundling behavior is important because the Expression Query Language (EQL) directly references object class names in a similar fashion to SQL referencing table names. There are certain...</summary>
   <author>
      <name>perren.walker</name>
      <uri>http://blogs.oracle.com/pwalker</uri>
   </author>
         <category term="Event Driven Architectures" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en-us" xml:base="http://blogs.oracle.com/pwalker/">
      <![CDATA[<p>This article highlights object bundling behavior within the Complex Event Processing Network.  Object bundling behavior is important because the Expression Query Language (EQL) directly references object class names in a similar fashion to SQL referencing table names.  There are certain instances where Expression Query Language will bundle objects into a MapEventObject.  You may or may not want this behavior to occur depending on what stage of event processing you are in. This article highlights the relationship among the Expression Query Language, POJO classes, and POJO classes bundled within a MapEventObject.  In this article I use POJO, Object and event interchangeably.</p>

<p>Events typically enter the Event Sever through Adapters and are forwarded on into the network.  If the event passes through a Complex Event Processing (CEP) node, the event is evaluated against the rules associated with that CEP node.  If one or more of the Expression Query Language (EQL) rules evaluates to true, then the object passes from the CEP node to the associated stream and then onto the next connecting node.</p>

<p>Let’s take a look at object behavior in the HelloWord demo application included with Oracle’s CEP Server 3.0.  In the HelloWorld demo application the CEP engine node is labeled helloworldProcessor.</p>

<p><br />
<img alt="HelloWorldEPN.gif" src="http://blogs.oracle.com/pwalker/HelloWorldEPN.gif" /></p>

<p><br />
In the helloworldProcessor the following EQL is executed:</p>

<p><strong>SELECT * from HelloWorldEvent retain 1 event</strong></p>

<p>The CEP node executing the above EPL will allow all of the HelloWorldEvent objects to pass onto the next node, perhaps an event bean or another CEP node.  In this case the HelloWorldBean: </p>

<p><strong>public class HelloWorldBean implements EventSink {</p>

<p>	@SuppressWarnings({ "unchecked" })<br />
	public void onEvent(List newEvents) throws EventRejectedException {</p>

<p>		for (Object event : newEvents) {<br />
			if (event instanceof HelloWorldEvent) {<br />
				HelloWorldEvent helloWorldEvent = (HelloWorldEvent) event;<br />
			<br />
			//Perform operations on the object here</p>

<p>			}<br />
		}<br />
	}<br />
}</strong></p>

<p>HelloWorldBean receives a list passed to the onEvent method.  As you iterate though the list you can check that the event is an “instanceof” the HelloWorldEvent, cast the object and manipulate it.</p>

<p>Objects will not always be forwarded like this based on their event type.  When performing a join or matching operation on a stream of events, the events will be bundled into a MapEventObject.  Additional Java language processing is required to get a the user defined java classes within the MapEventObject.  Let's take a look at some EQL examples and the object bundling that results.<br />
<strong><br />
select *  from (select * from SpotAndFowardEvent retain 1 events) spotForward, (select * from TheCostOfMoney retain 1 events)  costOM where  costOM.currencyPair = spotForward.currencyPair and  costOM.timePeriodDays = spotForward.timePeriodDays</strong></p>

<p>The EQL above joins two POJO streams of class type  SpotAndFowardEvent and TheCostOfMoney.  These two classes have attributes currencyPair and timePeriodDays; the join is performed when the attributes of the opposing classes are equal.  The output from this CEP process to the event stream is a MapEventObject.</p>

<p><strong>if (event instanceof MapEventObject) {<br />
  MapEventObject mEvent = (MapEventObject) event;<br />
	for (MapEventObject.Entry<String,Object> entry : mEvent.entrySet()){<br />
					System.out.println("getKey: : " + entry.getKey());<br />
                                   //Based on the key determine what object type to cast the value to<br />
	                           //Perform operations on the object here<br />
		}		<br />
}</strong></p>

<p>The Java code above shows you how to get at the objects located in the MapEventObject.  There is, however, another way of sending data out of the CEP node when joining two streams using the "INSERT INTO" clause of the EQL language. Here is the same join as the one performed by the EQL above, but in this example we are taking a third POJO object SpotAndFowardAndYield and populating its attributes with the attributes selected from the SpotAndFowardEvent and TheCostOfMoney POJOs.</p>

<p><strong>INSERT INTO SpotAndFowardAndYield(domesticCurrency, foreignCurrency, currencyPair, timePeriodDays) select spotForward.domesticCurrency, spotForward.foreignCurrency, spotForward.currencyPair, costOM.timePeriodDays from (select * from SpotAndFowardEvent retain 1 events) spotForward, (select * from TheCostOfMoney retain 1 events)  costOM where  costOM.currencyPair = spotForward.currencyPair and  costOM.timePeriodDays = spotForward.timePeriodDays</strong></p>

<p>Now instead of having to iterate through a MapEventObject you can just check to see if the event is an instance of SpotAndFowardAndYield and cast it to that type if you are so inclined. </p>

<p><strong>if (event instanceof SpotAndFowardAndYield) {<br />
				SpotAndFowardAndYield sfy = (SpotAndFowardAndYield) event;<br />
			<br />
			//Perform operations on the object here<br />
}</strong></p>

<p>So what is the best practice, either INSERT INTO a new object or manipulating a MapEventObject?  The answer is dependent on where your are in your event processing network.  If you are going to need to forward objects to another CEP node, INSERT INTO will provide a cleaner solution, mainly because the MapEventObject can not be manipulated very easily by the CEP engine.  If you are going to route one or more of the objects to separate streams, or if you are at the terminal node of your EPN then it might make sense to keep the objects separate within the MapEventObject. </p>

<p>When mapping a time/event based algorithm to an event processing network, it is important to keep this behavior in mind as it directly impacts the event object model within the network.  Building a visual EPN diagram and mapping out how you are going to deal with combining streams, before you code, can possibly save a great deal of  time.</p>

<p>  <br />
</p>]]>
      
   </content>
</entry>
<entry>
   <title>Oracle Complex Event Processing</title>
   <link rel="alternate" type="text/html" href="http://blogs.oracle.com/pwalker/2008/08/oracle_complex_event_processin.html" />
   <id>tag:blogs.oracle.com,2008:/pwalker//274.6052</id>
   
   <published>2008-08-26T18:56:13Z</published>
   <updated>2008-09-03T04:48:53Z</updated>
   
   <summary>This is a series of articles that, I hope, provide a bit of color to the workings of Oracle&apos;s Complex Event Processing. While event processing has been around for quite some time and is well established in industries such as...</summary>
   <author>
      <name>perren.walker</name>
      <uri>http://blogs.oracle.com/pwalker</uri>
   </author>
         <category term="Event Driven Architectures" scheme="http://www.sixapart.com/ns/types#category" />
   
   
   <content type="html" xml:lang="en-us" xml:base="http://blogs.oracle.com/pwalker/">
      <![CDATA[<p>This is a series of articles that, I hope, provide a bit of color to the workings of Oracle's Complex Event Processing.  While event processing has been around for quite some time and is well established in industries such as financial services, many non-financial industries are rapidly expanding their enterprise investment.  Consequently, I’m writing articles in two areas: 1) the behavior of the product and 2) event processing patterns. </p>

<p>Product behavior is important, impacting architectural decisions, developer productivity, and long term system maintainability.  These articles highlight internal workings not covered in documentation or perhaps not fully illuminated by use cases.</p>

<p>Event processing patterns can be combined in an event processing network (EPN) to solve a particular use case.  Interface, responsibility, construction, operation and extension patterns are established software development patterns.  With the addition of temporal processing, enabled by the Expression Query Language or Continuous Query Language, a new set of patterns emerge related to time based processing.  </p>

<p>It is my hope that these product articles, in combination with higher level design patterns will help you bring event driven applications into production quickly, with a high level of reliability.</p>

<p><a href="http://blogs.oracle.com/pwalker/2008/09/event_servers_a_disruptive_tec_1.html">1) Event Servers, A Disruptive Technology</a><br />
<a href="http://blogs.oracle.com/pwalker/2008/08/object_behavior_within_oracles.html"><br />
2) Object Behavior Within Oracle’s Complex Event Processing Server</a><br />
</p>]]>
      
   </content>
</entry>

</feed>
