Break New Ground
-
APIs
October 1, 2020
Integrating Oracle Functions and Oracle Streaming Service
Hello there,
Serverless compute is increasingly being preferred as the solution of choice for running application code on the cloud. Companies are moving to the Event-Driven era, where they care about to be real-time. Streaming Service is the natural choice when we talk about real-time and streaming data.
Combing the capabilities of Oracle's serverless compute platform, Oracle Functions, and Oracle streaming service, we achieve the best of both world advantages of being able to very easily produce and consume data to streams.
Journey to Event-Driven in a Serverless way
TL;DR
- Serverless compute is increasingly being preferred as the solution of choice for running application code on the cloud.
- Over the last several years, there has been a revolution, organizations must become real-time, to become real-time, they must be event-driven.
- Streaming Service is the natural choice when we talk about real-time and streaming data.
- Imagine the situation where you have a code that checks your streaming and trigger a function when your logic is valid.
- Did you know that you can create an Oracle Function that triggers a Streaming Service producer or consumer?
- Oracle Functions can call a Streaming producer and consumer.
- Oracle Streaming Service can trigger an Oracle Function.
In this article, we will build out a scenario of publishing and consuming messages from Oracle Streaming Service (Kafka) topic using Oracle Function (FN Project), and how Oracle Streaming Service trigger an Oracle Functions.
Pre-Requisites
- Access to Oracle Cloud
- OCI Functions
- OCI Streaming
The architecture of Oracle Functions call a Streaming Service Producer and Consumer
The architecture of Streaming service triggers Oracle Functions.
Function as a service (FaaS) is the ability to take a function and run it somewhere in the cloud, it’s the “compute” part of the serverless stack where you bring your own code. The function contains a bespoke logic block, It is then called via some kind of registry like an API gateway, or it is scheduled or triggered by a cloud-related event (i.e., data written to Oracle Object Store).
The function is expected to run for a period of time and then exit, with cloud vendors charging by the millisecond (and associated memory). The functions can have many instances running in parallel, and after the first “cold” start call, they are considered “hot.” The first call is where database connections (and the like) should be initialized. When calling via an API gateway, functions might be called synchronously to return a value.
There are many benefits to FaaS:
- Very cost-efficient
- Scalable
- Lightweight
- Decoupled
and of course, it's possible to create a function that calls a Streaming Service producer or a consumer.
Oracle Streaming Service (OSS) is Kafka compatible, and this means that you can use Kafka API including Kafka producer and Consumer.
We can divide into 3 scenarios;
A. Oracle Functions call Streaming Service producer
B. Oracle Functions call Streaming Service consumer
C. Oracle Streaming Service trigger Oracle Functions
Some ideas or use cases that integrating Oracle Functions and Streaming Service producer and consumer can be applied.
-
Read or write data from OCI infrastructure. (A or B)
- Start a producer or consumer from a rest call (A or B)
- Trigger a producer or consumer from an OCI event (A or B)
- Send some data to another cloud provider (A)
- Read on-premise data (B)
- When the stock market value grows 1% trigger the function to sell my stocks (C)
- When the sensor takes more than 5 seconds to send the new value to trigger a function to do some action (C)
- When the temperature value is bigger than a specific value (C)
- When the number of times that a motor start is bigger than 100 (C)
This is not an exhaustive list.
Nice, I understood all the benefits and I'm ready to create my function, now show me the code.
Let's code
scenario A
Oracle Functions call Streaming Producer Python example:
scenario B
Oracle Functions call Consumer Python example
scenario A & B
Oracle Functions call Streaming Service using Java Oracle SDK example
full code here
scenario A & B
Oracle Functions call Streaming Service using Java Micronaut example
You can use API Gateway, to expose a REST interface to trigger the function.
scenario C
This part uses Kafka Connect.
Oracle Streaming Service trigger Oracle Functions
The method "triggerFn" is where you should add your business logic to trigger the function.full code here
Links
Getting Started With Oracle Streaming Service OSS
Oracle Streaming Service Producer & Consumer
Getting Started with API Gateway
Micronaut OCI Functions GitHub
Photo by Taylor Vick on Unsplash