Introduction  

High-Performance workloads such as AI/ML and Oracle database push the boundaries of network scale, reliability and performance.  In addition to these expectations from customer workloads, cloud providers need to also ensure strong security, low cost, and power efficiency. 

Watch the video to learn from the authors in this in-depth discussion of how OCI builds resilient networks for AI workloads at scale.

As the scale of workloads increase, even the low probability failures become more common.  Furthermore, the tightly synchronized nature of AI workloads implies that blast radius of failures is quite large, sometimes to the scale of the entire workloads. Oracle Cloud Infrastructure, OCI had previously launched multi-planar fabrics to address this reliability requirement. These fabrics provide n-ways redundancy for end hosts and go a long way in ensuring workload reliability. 

In a previous blog post, we discussed how Multipath Reliable Connection within OCI’s Multiplanar networks achieve high reliability and high performance. In this blog, we present the next wave of innovations OCI has made on behalf of our customers. These innovations are aimed at providing higher reliability and performance for our customer workloads, including AI/ML workloads. We are sharing our next-gen innovations:  

  1. Oracle Acceleron High-Performance Fabric, and  
  1. Oracle Multi-Planar Fabric using Oracle Collective Communications Library Plugin, OCCL Plugin 

Oracle Acceleron High-Performance Fabric 

OCI’s front-end network operates at massive scale and supports a diverse mix of workloads, including Oracle Database, customer compute, block and object storage, Oracle SaaS and OCI services, third-party services, OCI backbone connectivity, Internet access, and customer on-premises connectivity. 

Remote Direct Memory Access, RDMA moves data directly between memory on different machines with minimal CPU involvement. By bypassing much of the operating system and software processing in a conventional transfer, it can reduce latency, sustain high throughput, and leave more CPU capacity for applications. These benefits are useful beyond GPU-to-GPU communication in a cluster, including for storage access and cross-cluster transfers. 

Deploying RDMA is considered a challenge in such a heterogenous environment. Across a mixed-workload network, RDMA is highly sensitive to congestion, latency, and packet loss. In a large, shared network: 

  • Different workloads generate very different traffic patterns; short, bursty storage and database flows alongside long-running compute or internet flows. 
  • RDMA transport expects extremely low loss; even modest congestion can sharply reduce performance or trigger inefficient recovery behavior. 
  • A congestion-control mechanism must protect latency-sensitive applications without allowing high-bandwidth flows to dominate shared links. 
  • The network spans many paths, services, and connection types, including customer and internet-facing traffic, so conditions are less predictable than in a dedicated cluster. 
  • The design must work safely and fairly on a massive scale, without requiring each workload or customer to be specially tuned. 

The core challenge is maintaining predictable, high-performance RDMA while sharing the same infrastructure with workloads that have competing and often conflicting network requirements. OCI has addressed it with new capabilities in Oracle Acceleron SmartNICs. 

Figure 1: Oracle Acceleron High-Perforamnce Fabric
Figure 1: Oracle Acceleron High-Perforamnce Fabric

Oracle Acceleron High-Performance Fabric requires minimal support for QoS from the network substrate, increases workload throughput while reducing workload latency – all the while providing standard RDMA CPU and Kernel bypass on the end hosts.  

One challenge in sharing the front-end fabric is that RDMA typically uses rate-based congestion control, while TCP commonly uses window-based congestion control. Depending on the traffic pattern, one control loop can overwhelm the other. To help preserve fairness, Oracle Acceleron SmartNICs use hierarchical token buckets (HTB) to track bandwidth consumption for RDMA and non-RDMA traffic. A parent bucket represents shared bandwidth, with child buckets for each traffic class. Tokens replenish at configured rates and are consumed as traffic arrives. As available tokens decline, the Oracle Acceleron SmartNIC probabilistically marks ECN-capable packets or drops packets that cannot respond to ECN. This produces a gradual congestion signal. HTB is work-conserving: each traffic class can borrow unused bandwidth from the parent, allowing either class to use the available bandwidth when the other is absent. 

Figure 2: Figure 2. Hierarchical Token Buckets (HTB) Congestion Signaling 
Figure 2: Hierarchical Token Buckets (HTB) Congestion Signaling 

HTB also addresses incast at the connection between a top-of-rack switch and a SmartNIC. As tokens in the parent bucket begin to deplete, it signals congestion through probabilistic marking or drops before the incast queue builds further. This helps protect latency while multiple senders direct traffic toward the same destination. 

For congestion within the fabric, OCI uses One-Way-Delay Congestion Control (OWD-CC) in Acceleron SmartNICs. Measuring forward-path delay avoids the false congestion signal that a round-trip measurement can produce when only the reverse path is congested. When the SmartNIC detects elevated delay for a flow, it probes three alternate paths and moves the flow if it finds a better one. If no better path is available, it probabilistically signals congestion with ECN marking for traffic that responds to ECN, or with packet drops otherwise.  

OCI’s front-tend RDMA supports all Queue Pair (QP) types and verbs, allowing customers to run their existing applications without any changes. These capabilities allow customers to run storage, AI, and general-purpose traffic on the same frontend network while preserving performance and isolation. 

Multi-Planar Fabrics using Oracle Collective Communications Library Plugin 

Large AI clusters are engineered around a simple constraint: synchronized jobs only move as fast as the slowest link in the chain, e.g. a congested or flaky network path could slow down a single flow and lead to a workload-wide slowdown. When thousands of GPUs participate in collectives, a single congested link, failed path, or unlucky flow collision can stretch tail latency and waste expensive compute. Traditional datacenter networks tend to fall short of achieving full fabric utilization because the standard paradigm of relying on Equal-Cost Multiple-Path (ECMP) forwarding to load-balance traffic leads to uneven link utilization.  This happens because ECMP “pins” flows (e.g. RDMA QPs) to network paths all the while not knowing the size (bandwidth) of flows – thus resulting in uneven load-balancing of flows over parallel paths. 

Failures and congestion in the network can cause jobs to stall, restart, or take longer to complete, leaving expensive GPU capacity underutilized. AI workloads in particular need a more resilient and efficient network.  OCI multi-planar fabric has been designed to meet that very need.  

Muti-planar fabrics imply the use of multi-port NICs to connect to multiple disjoint fabrics.  This requires specialized functionality on the NICs, which isn’t always feasible.  We have been working on removing this constraint and offering multi-planar networks for general AI/ML workloads.  We have achieved that goal, and it comes in the form of Oracle Collective Communications Library plugin, OCCL. 

Figure 3: Multi-Planar Network Design using Oracle Collective Communication Library (OCCL)
Figure 3: Multi-Planar Network Design using Oracle Collective Communication Library (OCCL)

OCCL allows customers to connect to multi-planar fabrics and thus gain higher workload reliability.  OCCL plugin provides superior workload performance by:  

  1. distributing traffic across multiple QPs, 
  1. sending more traffic over QPs that experience higher throughput, and 
  1. actively moving traffic over to less congested, or uncongested, paths in the fabric. 

OCCL is a CCL network plugin that supports commonly adopted collective-communication libraries from market leading companies. AI frameworks use CCLs to coordinate data exchange among GPUs. For inter-node communication, these CCLs use a network-plugin interface to translate collective-communication transport requests into NIC and network-fabric transfers. We have built key network functionality, including failover, recovery, and congestion-aware load balancing, into the plugin.

Figure 4: OCCL Built-in Failover and Recovery
Figure 4: OCCL Built-in Failover and Recovery

Moving this functionality into the CCL layer, decouples it from network hardware and firmware. The capabilities built into the plugin do not have dependency on network topologies either. It works well for both single-planar and multi-planar networks. OCCL is designed to detect network congestion and to intelligently load-balance traffic across the available fabric planes, selecting less-congested planes and less-congested paths, thus improving workload performance.

It also supports port-level and NIC-level traffic failover, transparently steering collective communication to another available port or NIC after a link or NIC failure. This allows distributed training to continue without requiring a job to restart due to link failures.   

OCCL implements congestion-aware, message-level and chunk-level traffic load-balancing through the standard CCL network-plugin interface. It dynamically assigns collective-communication messages across standard Reliable Connection (RC) QPs based on observed congestion and health.  It can also move traffic away from failed ports, including failed NIC ports or NICs. OCCL is transparent to CCL-based customer workloads.  

We believe OCCL is simpler for customers to use because it does not require customers to resort to complex routing controllers to select paths in the fabric or to detect any path changes. 

Conclusion 

OCI Acceleron is a set of innovative technologies aimed at improving the reliability, performance, and scale of customer workloads.  Key next-gen innovations include RDMA on OCI front-end network and OCI CCL plugin. These technologies are aimed at adding security, reliability, and resilience to our customers. OCI is committed to relentless innovation on behalf of the customer.   

Key Takeaways 

  • Oracle Acceleron High-Performance Fabric – Front-end RDMA: OCI brings high-performance RDMA to shared networks while maintaining predictable latency, throughput, and workload isolation.  
  • Oracle Multi-Planar Fabric using OCCL: Congestion-aware load balancing and automatic failover keep AI training jobs running efficiently despite network congestion or failures.  
  • Advanced resilience: Moving intelligence into the collective-communications layer reduces dependence on specialized network hardware and complex routing controls.