With the rapid evolution of event-based architecture and the proliferation of connected devices— from ubiquitous mobile devices to IoT devices in industrial automation—customers must deal with a continuous flow of high-volume data that can strain the resources of their on-premises systems. Oracle customers building modern cloud native applications on Oracle Cloud Infrastructure need a fully managed solution to help them ingest large amounts of continuous data (sometimes from multiple sources) and make that data available for immediate processing.

Today we are proud to announce the general availability of the Oracle Cloud Infrastructure Streaming service in all Oracle Cloud Infrastructure commercial regions. The Streaming service provides a fully managed, scalable, and durable storage solution for ingesting continuous, high-volume streams of data that you can consume in real time. Streaming can be used for messaging, ingesting high-volume data such as application logs, operational telemetry, web click-stream data, and other use cases in which data is produced and processed continually and sequentially in a publish-subscribe messaging model.

As part of Oracle Cloud Infrastructure, Streaming is integrated with Identity and Access Management (IAM), which enforces fine-grained security rules through access control policies. Additionally, all streaming data is encrypted both at rest and in motion.

Streaming pricing is intuitive, simple, and elastic; you pay only for what you use, which makes it attractive for workloads with large spikes (no more paying for provisioned resources to handle application spikes). In addition to supporting major functionality such as producing to a stream and consuming from a stream, the Streaming service simplifies application development by providing managed consumers, which offloads load balancing, coordination, and offset management.

Streaming is accessible through the Oracle Cloud Infrastructure Console, SDKs, CLI, and REST API, and provides Terraform integration.

Getting Started

Getting started with the Streaming service is straightforward in the Oracle Cloud Infrastructure Console and by using the REST API. The following steps show how to create a stream, produce messages to the stream, and consume messages from the stream.

  1. From the console main menu, navigate to the Streaming section (Analytics > Streaming) in the appropriate compartment.
  2. Create a new stream by clicking Create Stream. Alternatively, you can call the CreateStream REST API operation.

  3. Specify a name for the stream, the number of hours to retain messages, and the number of partitions based on the throughput requirement. Then, click Create Stream.


     
  4. After the stream is created, you can start producing messages to your stream. In the Console, click Produce Test Message. Or, using the REST API, you can use the PutMessages operation and pass a key and payload to produce data to a stream.


     
  5. Enter a sample payload, and click Produce.


     
  6. After you receive the acknowledgement, you can start consuming messages from the stream. From the Console, you can click Refresh to retrieve the produced messages. Using the REST API, you first create a cursor by using the CreateCursor operation and then calling GetMessages by passing the cursor.


     

We want you to experience this new service and all the enterprise-grade capabilities that Oracle Cloud Infrastructure offers. It’s easy to try it with our US$300 free credit. For more information, see the Oracle Cloud Infrastructure Getting Started, Streaming overview, Streaming documentation, and FAQ.

Be on the lookout for announcements about additional features and capabilities, including Kafka compatibility for read and write, Kafka Connect compatibility, private virtual cloud network (VCN) endpoints, and Object Storage archival for additional retention of streaming data. We value your feedback as we continue to enhance our offering and make our service the best in the industry. Let us know how we can continue to improve or if you want more information about any topic. We are excited for what’s ahead and are looking forward to building the best streaming data platform.