X

Break New Ground

Integrating Oracle Functions and Oracle Streaming Service

Igor Aragao de Souza
Principal Big Data Consultant

Hello there,

Serverless compute is increasingly being preferred as the solution of choice for running application code on the cloud. Companies are moving to the Event-Driven era, where they care about to be real-time. Streaming Service is the natural choice when we talk about real-time and streaming data.

Combing the capabilities of Oracle's serverless compute platform, Oracle Functions, and Oracle streaming service, we achieve the best of both world advantages of being able to very easily produce and consume data to streams.

Journey to Event-Driven in a Serverless way

TL;DR

  • Serverless compute is increasingly being preferred as the solution of choice for running application code on the cloud.
  • Over the last several years, there has been a revolution, organizations must become real-time, to become real-time, they must be event-driven.
  • Streaming Service is the natural choice when we talk about real-time and streaming data.
  • Imagine the situation where you have a code that checks your streaming and trigger a function when your logic is valid.
  • Did you know that you can create an Oracle Function that triggers a Streaming Service producer or consumer?
  • Oracle Functions can call a Streaming producer and consumer.
  • Oracle Streaming Service can trigger an Oracle Function.

In this article, we will build out a scenario of publishing and consuming messages from Oracle Streaming Service (Kafka) topic using Oracle Function (FN  Project), and how Oracle Streaming Service trigger an Oracle Functions.

Pre-Requisites

  • Access to Oracle Cloud
  • OCI Functions
  • OCI Streaming


The architecture of Oracle Functions call a Streaming Service Producer and Consumer

 


The architecture of Streaming service triggers Oracle Functions.


Function as a service (FaaS) is the ability to take a function and run it somewhere in the cloud, it’s the “compute” part of the serverless stack where you bring your own code. The function contains a bespoke logic block, It is then called via some kind of registry like an API gateway, or it is scheduled or triggered by a cloud-related event (i.e., data written to Oracle Object Store).

The function is expected to run for a period of time and then exit, with cloud vendors charging by the millisecond (and associated memory). The functions can have many instances running in parallel, and after the first “cold” start call, they are considered “hot.” The first call is where database connections (and the like) should be initialized. When calling via an API gateway, functions might be called synchronously to return a value.

There are many benefits to FaaS:

  • Very cost-efficient
  • Scalable
  • Lightweight
  • Decoupled

and of course, it's possible to create a function that calls a Streaming Service producer or a consumer.

Oracle Streaming Service (OSS) is Kafka compatible, and this means that you can use Kafka API including Kafka producer and Consumer.

We can divide into 3 scenarios;

A. Oracle Functions call Streaming Service producer
B. Oracle Functions call Streaming Service consumer
C. Oracle Streaming Service trigger Oracle Functions

 

Some ideas or use cases that integrating Oracle Functions and Streaming Service producer and consumer can be applied.

  • Read or write data from OCI infrastructure. (A or B)

  • Start a producer or consumer from a rest call (A or B)
  • Trigger a producer or consumer from an OCI event (A or B)
  • Send some data to another cloud provider (A)
  • Read on-premise data (B)
  • When the stock market value grows 1% trigger the function to sell my stocks (C)
  • When the sensor takes more than 5 seconds to send the new value to trigger a function to do some action (C)
  • When the temperature value is bigger than a specific value (C)
  • When the number of times that a motor start is bigger than 100 (C)

This is not an exhaustive list.

 

Nice, I understood all the benefits and I'm ready to create my function, now show me the code.

Let's code

scenario A

Oracle Functions call Streaming Producer Python example:


scenario B

Oracle Functions call Consumer Python example


scenario A & B

Oracle Functions call Streaming Service using Java Oracle SDK example

full code here


scenario A & B

Oracle Functions call Streaming Service using Java Micronaut example

Announcing the Micronaut Oracle Cloud Module for Simple SDK Integration and Performant Serverless Functions

 

Oracle Python SDK

Oracle Java SDK

 

You can use API Gateway, to expose a REST interface to trigger the function.

API Gateway Menu

scenario C

This part uses Kafka Connect.

Oracle Streaming Service trigger Oracle Functions

The method "triggerFn" is where you should add your business logic to trigger the function.

full code here

 

Links

OCI Functions

Overview of OCI Functions

Micronaut OCI Functions

OCI Streaming

Getting Started With Oracle Streaming Service OSS

Oracle Streaming Service Producer & Consumer

OCI API Gateway

Getting Started with API Gateway

Oracle Functions GitHub

Micronaut OCI Functions GitHub

 

 

Photo by Taylor Vick on Unsplash

 

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.