We’re announcing support for OpenSearch v2.19 which includes additional functions for search and observability use cases, helps enhance performance and ease of use, and adds several new features for machine learning (ML) and generative AI applications in our fully managed cloud service OCI Search with OpenSearch. Additionally, we are also announcing the Limited Availability of OpenSearch Push and OpenTelemetry connectors for data ingestion to complement the pull connectors that have been introduced in 2.18. OpenSearch, 2.19 will be also a jump release for 3.0 which will be released toward the end of the year.

Key Use Cases for OCI Search with OpenSearch:

  • AI-powered observability: Advanced AI search capabilities for log analytics and monitoring combined with AI assistant toolkit.
  • AI and machine learning for search applications: Seamless ML integration for AI-driven search.
  • Semantic and conversational search: Enhanced vector search for intuitive interactions.
  • Security analytics: Improved insights for security monitoring.

Observability AI-powered

This release includes new capabilities for observability and log analytics tasks.

Adding OpenSearch Push and Open Telemetry connectors via OCI Connector Hub

This is the logical step after the Data Prepper Pull connectors released in the OCI OpenSearch 2.18 timeframe. We are adding OpenSearch Push and OpenTelemetry connectors for data ingestion and supporting OCI Logging as a data source via OCI connector Hub.

Illustration of the data pipeline to OpenSearch using OCI Data Prepper
Figure 1: Illustration of the data pipeline to OpenSearch using OCI Data Prepper

 

Improved anomaly detection

The Anomaly Detection feature, originally available in previous versions, has been enhanced in version 2.19. This update allows users to configure detection settings to identify specific data patterns, resulting in improved catch rates and greater flexibility in anomaly detection.

AI and machine learning improvements for your search applications

There are several enhancements in 2.19, but the most important one is the introduction of OpenSearch Flow — this new feature enhances data processing and workflow automation within OpenSearch, providing users with greater efficiency and flexibility.

Simplified application development using ingest and search pipelines in OpenSearch Dashboards

OpenSearch Flow now empowers developers to create advanced AI applications directly within OpenSearch Dashboards. Previously, development was limited to the command-line interface (CLI), but now both ingestion and retrieval-augmented generation (RAG) pipelines can be configured using an intuitive graphical interface. To further simplify development, a variety of pre-built templates are available for different use cases.

OpenSearch Flow interface
Figure 2: OpenSearch Flow interface

Read and write operations from pluggable storage

In OpenSearch 2.18, users could leverage pluggable storage for reading vector data structures. With this latest version, pluggable storage now supports both read and write operations, offering greater flexibility and efficiency in data management.

Binary vectors for the Lucene engine

According to OpenSearch, Binary vectors enable customers to reduce memory and storage usage by over 80% compared to FP32 vectors. As a result, OCI hardware configurations can be smaller without any performance degradation. OpenSearch 2.19 supports Lucene binary vectors alongside Faiss engine binary vector support, offering more flexibility for applications utilizing vector search.

Cosine-based similarity search in conjunction with Faiss vector engine

Customers can now leverage the OpenSearch k-NN plugin with cosine similarity in the Faiss engine for both k-NN and radial search, enabling similarity searches based on cosine distance.

Hybrid search results augmented with reciprocal rank fusion

Customers can now leverage reciprocal rank fusion (RRF) to enhance results. By basing rankings on document positions rather than scores, RRF offers a potentially improved ranking approach.

Semantic and conversational search improvements

There is a new functionality for search too in this release

Search operations can be improved utilizing a new template query type

Customers can now create queries with placeholder variables, functioning like templates. These placeholders remain unfilled until the customer submits the request.

More efficient performance monitoring

In OpenSearch 2.19, customers can take advantage of the new query insights dashboards, which aggregate data from the top N queries collected by the Query Insights plugin. This enables a comprehensive analysis of historical queries and provides a drill-down function for more detailed exploration. Additionally, the configuration is displayed on the overview page, aiding analysis and potential modifications.

Get Started Today

With these enhancements, OCI Search with OpenSearch continues to offer enterprise-grade capabilities for large-scale, distributed search and analytics workloads. This update comes at no additional cost, with transparent OCI pricing based on node count rather than core usage.

Are you ready to explore these new features? Head to the Oracle Cloud Console and start upgrading today.

For more information, see the following resources: