Exadata System Software 25ai release 25.1

The Exadata Product Management and Engineering teams are excited to announce the availability of the latest release of Exadata System Software—Exadata System Software 25ai, release 25.1! Building on Exadata System Software 24ai and prior releases, the 25.1 release introduces significant new capabilities. It enhances existing capabilities to cement Exadata’s position as the best platform for running Oracle Database on-premises or in the cloud. Read on for a summary of the AI, Exascale, Performance, High Availability, Security and Manageability highlights.

AI Smart Scan Adaptive Top-K Filtering, Vector Distance Projection, and BINARY vectors

Exadata System Software 24ai and Oracle Database 23ai introduced new ground-breaking AI Vector Search capabilities that enable customers to build new applications with, or add semantic search functionality to, existing applications and take advantage of new artificial intelligence technologies and patterns. Exadata uniquely accelerates these use cases by offloading AI Vector Search queries from the database to the storage servers, leveraging the scale-out architecture of Exadata, and unleashing superior hardware components and unique software algorithms and optimizations. We call this AI Smart Scan.

Fundamental to AI Smart Scan is the same core concept as Exadata Smart Scan – move SQL processing to the data instead of data to the SQL process. In Exadata System Software 24ai, we did precisely this. On a non-Exadata system, a vector search query requires vast quantities of vector data to be sent from storage to the database to calculate the distance between each vector before discarding significant amounts of data and finally arriving at the top-K most relevant result. Exadata transparently offloads the query and automatically parallelizes distance computation between vectors on the storage servers, sending only the Top-K vectors and related business data to the database server for final processing. The resulting 30x performance improvement speaks for itself.

Adaptive Top-K Filtering

In Exadata System Software 25.1, we further optimized the Top-K calculation so that each storage server maintains a running Top-K to reduce further the amount of data sent back to the database from the storage servers.

So, how do we calculate the Top-K? Let’s start with the prior behavior. Each storage server determines the Top-K vectors for each scanned on-disk region. These Top-K results and other columns requested by the query are sent to the database server from all the storage servers. Coming back to our example from above, for every region on disk scanned, a Top-K for that region is determined. That data is sent back to the database to merge, and then the final Top-K is determined.

In this release, rather than sending all this data to the database – much of which becomes irrelevant as the global Top-K is determined – each storage server maintains a local running Top-K, updated to reflect only vectors that improve the existing set. In other words, tracking the Top-K for a query at the storage server fllters the irrelevant data at the storage layer, reducing the amount of data returned to the database by up to 4.69x.

AI Smart Scan Adaptive Top-K Pruning animation

Vector Distance Projection

Rather than the database consuming significant CPU resources by recalculating the distance between the Top-K results for each storage server and the initial input vector, Exadata System Software 25.1 projects (returns) the vector distance already calculated by the storage server to the database. Vector Distance Projection leads to up to 4.6x faster Vector Search queries and up to 24x less data sent to the database; we don’t need to send vectors to the database if they don’t need to be processed!

BINARY Vectors

Database 23ai Release Update 5 (23.5) adds INT8 and BINARY vector dimension formats to the existing FLOAT32 and FLOAT64 vector dimensions formats added in the initial Database 23ai release. INT8 and BINARY vectors are more storage-efficient, requiring up to 32x less storage than FLOAT32 and enabling up to 40x faster distance computation while maintaining an accuracy of well over 92% . In other words, using INT8 or BINARY vectors saves space, and queries run faster with almost the same accuracy as other dimension formats.

CREATE TABLE SUPPORT_INCIDENTS (id              NUMBER,
                                incident_text   VARCHAR2(1000),
                                incident_vector VECTOR(1024, BINARY));

On Exadata, AI Smart Scan further accelerates the AI query performance of vectors with INT8 and BINARY dimension formats by up to 8x and 32x, respectively, while maintaining accuracy above 92% compared to FLOAT32.

Comparison of FLOAT32, INT8 and BINARY Vector Dimension Formats

Exascale Free Space Management

Exadata Exascale is our new intelligent data architecture for the cloud built into Exadata System Software. A key component of this new architecture is decoupling storage management from database servers and managing data across and within the storage servers. By managing extents for all clusters and databases holistically, Exascale can reduce the free space required to successfully rebalance data in the event of a failed disk drive or capacity-optimized flash device, from as high as 29% to as low as 3%, depending on the quantity and type of storage servers in the storage pool. This is a significant improvement for machine administrators who must balance capacity, availability, and cost when sizing their Exadata platforms.

Exascale Storage Pools reserve as little as 3% as more storage is added

On Exadata, each storage server is a failure group. For configurations with three storage servers, enough free space is required within the storage server with a failed disk or flash device to rebuild the failed device extents and maintain redundancy. However, after adding a fourth server (or more), data on a failed disk can be rebalanced to a different server, increasing the number of target partner disks and significantly reducing the free space required to ensure the operation is successful. The reserved free space needed includes files stored in high and normal redundancy as Exascale Storage Pools enable both concurrently.

Exascale Volume Clone Enhancements

Two-step Exascale Volume Thin Clone

Introduced in Exadata System Software 24ai as part of Exascale, Exascale Volumes are a new capability in Exadata that enables block devices to be created on the shared Exadata storage servers and mounted to Exadata database servers, enabling the use of file systems such as ACFS, XFS, and ext4. In addition to file system storage, Exascale Volumes are also used to allow for high-density virtual machine configurations – up to 50 per X10M database server on-premises – by storing the VM image files on shared storage. The shared Exadata Exascale storage dramatically increases the usable capacity for filesystems and VM images and improves performance as Exascale Volumes are RDMA enabled!

Two-step Exascale Volume Thin Clone Administrators can create read-only snapshots and full or thin read/write clones from an existing Exascale Volume. A snapshot is a prerequisite to creating a volume clone. Creating a read-only snapshot before creating volume clones is useful when multiple clones are needed simultaneously from the same point in time.

Beginning in Exadata System Software 25.1, creating an Exascale Volume Clone automatically and transparently creates the required read-only snapshot, reducing operational steps when a single volume clone is needed.

Automatic Acceleration of ASM Rebalance

Automatic ASM Rebalance Management Framework

ASM on Exadata has long given administrators the ability to tune or set the speed at which ASM performs rebalance operations. ASM Rebalance occurs when a disk is replaced or storage is added or removed. As the name implies, it rebalances extents across the storage servers so that each disk has a balanced amount of data. Determining the “correct” rebalance power – as it describes how powerful the rebalance operation is – requires ongoing monitoring and understanding of the database workloads. The higher the power, the faster the rebalance runs and the sooner redundancy is restored. The inverse is also true; the lower the power, the slower the rebalance runs, minimizing the impact of database I/O performance.

Beginning with this release, Exadata System Software, in combination with 23ai Grid Infrastructure, continuously monitors database I/Os and automatically adjusts the ASM rebalance power higher or lower to maximize and, therefore, reduce the time taken to rebalance extents without impacting database performance. By automatically and dynamically adjusting the rebalance power, restoring redundancy after disk failure completes up to 3.6x faster, resync operations after storage server updates are up to 2x faster, and adding a storage server is up to 2.6x faster.

Oracle Linux with the Unbreakable Enterprise Kernel

Oracle Linux with the Unbreakable Enterprise Kernel (UEK) 7 debuts on Exadata in this release, bringing with it many significant improvements, including a better scheduler enabling up to 20% faster analytical query performance, and the capability to load the executable instruction portion of the Oracle Database 23ai binary into huge pages resulting in up to 12% performance boost for OLTP. Exadata is known for running the most critical database workloads, and we take a very cautious approach when upgrading the operating system and kernel on the platform. In this case, the OS remains on Oracle Linux 8 Update 10, and the kernel is upgraded to the latest UEK 7 Update 3. This approach ensures that our customers and 3rd party application vendors have ample time to certify their applications, tools, and utilities on the OS. By upgrading the kernel, we receive the latest features and fixes and lay the foundation for future innovations we have planned without requiring customers to upgrade the Oracle Linux release more frequently than necessary.

For all Exadata systems with an RDMA over Converged Ethernet (RoCE) network (X8M and newer), the kernel is upgraded to UEK 7 for bare metal servers, KVM Hosts, and KVM Guests. X8 and earlier systems with Inifiband networks, Dom0 domains (database servers deployed for virtualization) are upgraded with UEK 6 on Oracle Linux 7, and bare metal servers and virtual machines (DomU) continue to run UEK 6 on Oracle Linux 8.

Maintaining Additional Linux Packages while updating Database Servers

Customers install additional software on their database servers, such as backup utilities, monitoring agents, and security applications. These packages often require additional Linux RPMs (dependencies) that are not included in the lightweight Oracle Linux distribution we ship with Exadata. Customers can install the necessary RPMs using standard Linux utilities to resolve such dependencies. During database server updates, the additional packages needed for the 3rd-party software often also require updates to resolve dependency conflicts. Before this release, customers uninstalled these additional packages and software to resolve such conflicts, updated the database servers using patch manager (patchmgr), and then reinstalled the additional packages and software.

Starting with Exadata System Software 25.1, administrators can manage the additional Linux packages with patchmgr during both the precheck and upgrade phases. This enables patchmgr to resolve dependencies and conflicts and avoids the need for customers to remove software required in their environments.

patchmgr example syntax and additional repo contents

Faster Storage Server Upgrades

By optimizing and parallelizing many operations, we’ve increased the speed at which storage server updates occur by up to 1.45x compared to previous releases. This speeds up the update and reduces storage server downtime, which accelerates the resync phase, as less data must be replayed on each server.

Centralized Management Scalability

The dcli utility facilitates centralized management across Exadata servers. It enables administrators to execute Linux commands, including image info, cellcli, and dbmcli, across multiple servers simultaneously. In this release, dcli seamlessly scales to cloud-scale Exadata deployments, enabling hundreds to thousands of servers to be managed from a single dcli execution.

Increased RoCE Network Resilience Coverage for Higher Availability

Introduced in Exadata System Software 24ai, exaportmon is a new process on every Exadata database and storage server that continuously monitors Exadata’s private network. Each database and storage server has dual-100GbE/s RDMA over Converged Ethernet (RoCE) ports and private IP addresses that must be highly available to ensure cluster stability and avoid outages. A misconfiguration or outage of a RoCE leaf switch, to which each server is connected, may lead to a stalled port that appears online but on which network traffic cannot flow. Exaportmon automatically detects stalled switch ports and moves the IP address of the stalled port to the other active RoCE port on the database or storage server, enabling network traffic to flow. Exaportmon automatically moves the IP address back to its original port after the switch port issue has been resolved.

In this release, exaportmon monitors for online RoCE switch ports with network traffic flowing that may send corrupted network packets (or exhibit other symptomatic behavior). In this case, Exaportmon will identify the sick switch and shut off the port(s) associated with it on the database or storage server. This continued monitoring and increased failure scenario detection and avoidance increase Exadata’s overall availability.

RDMA Network Updates

The private network in Exadata is vital to its overall architecture, performance, stability, and security. The RDMA over Converged Ethernet (RoCE) and Infiniband switches for X7 and newer deployments have been updated to the following releases.

Exadata X8M and newer (X8M-X10M) – Cisco RoCE switch updated to NX-OS 10.3.6
Exadata X7 and X8 deployments – InfiniBand switch updated to 2.2.17-2

As with all updates on Exadata, switch updates can and should be applied in a rolling manner to maintain high availability.

Exadata Cache Statistics (ecstat) Improvements

ecstat flash statistics screen shot The Exadata Cache Statistics (ecstat) utility, first available in Exadata System Software 24ai (24.1), provides fine-grained and real-time insight into the Exadata Flash Cache in the Exadata storage servers. Similar in execution to Linux utilities such as top or iostat, ecstat breaks down client database I/Os by media type and I/O reason, giving visibility into workload, throughput, and performance.

New in Exadata System Software 25.1, ecstat provides insight into Flash Cache Bypass Reasons, Top Flash Cache Space Consumers, and Top I/O Consumers.

New ecstat insights

ecstat can be executed directly on storage servers for real-time monitoring and is automatically included in the data Exawatcher collects.

Secure-by-Default

Exadata System Software 20.1 introduced a fantastic security capability: Exadata Secure RDMA Fabric Isolation (Secure Fabric for short). Secure Fabric is analogous on X8M, and newer Exadata deployments to the Infiniband Partitioning are available on X8 and earlier machines. It ensures that the private RDMA over Converged Ethernet (RoCE) network cannot be used to allow lateral unauthorized access between virtual machine clusters. Secure Fabric significantly improves security in consolidated and cloud environments where different departments or customers use the same shared physical infrastructure and have strong isolation and security requirements.

With Exadata System Software 25.1 and the October 2024 Oracle Exadata Deployment Assistant (OEDA), Secure Fabric is automatically selected for new X8M and newer virtualized deployments. It brings new on-premises deployments into line with X8M and newer Exadata deployments in the cloud, which are ALL deployed with Secure Fabric configured.

Hardware, Grid Infrastructure, and Database Minimum Requirements

Hardware

Exadata System Software 25.1 is available for all X7 (X7-2 and X7-8) and newer Exadata Database Machines and Storage Expansion Racks.

Grid Infrastructure and Database

Exadata System Software 25.1 has the following minimum software requirements for Oracle Database and Grid Infrastructure:

Minimum supported Oracle Grid Infrastructure releases:
- Release 19c: Version 19.23, April 2024 Release Update (RU)
- Release 23ai: Version 23.5 (July 2024)
Minimum supported Oracle Database releases:
- Release 19c: Version 19.21, October 2023 Release Update (RU)
- Release 23ai: Version 23.5 (July 2024)