Introduction

Large-scale artificial intelligence (AI) training depends on collective communication among hundreds or thousands of graphics processing units (GPUs). As clusters grow, a congested network path, failed link, or failed network interface card (NIC) can interrupt collective communication, lower GPU utilization, and force a training job to stall or restart. For long-running training jobs, the resulting lost work and recovery time can be significant.

Oracle Cloud Infrastructure (OCI) is introducing Oracle Collective Communications Library (OCCL) network plugin, a collective-communication network plugin designed for distributed AI workloads on OCI-supported GPUs and remote direct memory access (RDMA)-enabled AI infrastructure. OCCL works with NVIDIA Collective Communications Library (NCCL) and AMD Radeon Collective Communications Library (RCCL) to provide congestion-aware traffic distribution across available network planes and automatic recovery from supported link and NIC failures. It does this through the standard NCCL/RCCL network-plugin interface, without requiring changes to customer training applications, AI frameworks, or collective-communication calls.

The initial generally available OCCL network plugin has been released. It works with both NCCL and RCCL on supported RDMA over Converged Ethernet version 2 (RoCEv2) and InfiniBand cluster-network configurations. For supported version information, refer to the document.

Why collective communication needs a more resilient network

Distributed training repeatedly exchanges model parameters, gradients, and other data among GPUs. These collective operations are sensitive to network congestion and failures because the overall step often waits for the slowest participating path.

OCI GPU clusters use RDMA networks that can be deployed with a multi-planar architecture, in which network capacity is provided through multiple independent planes. This design creates path diversity and helps isolate failures. To gain the full benefit, collective traffic must use capacity across the available planes and respond to congestion or failures without interrupting the training job.

Without congestion-aware traffic distribution, collective traffic can concentrate on some paths while others are underused. This uneven utilization can increase latency, reduce training throughput, and leave GPU capacity idle. On the other hand, a link or NIC failure can also interrupt collective communication and cause a job to stall, restart, or take longer to complete.

What the OCCL network plugin does

The OCCL network plugin places the needed networking intelligence for congestion control and link/NIC failure recovery in the collective-communication transport layer. AI frameworks continue to use NCCL or RCCL, and the collective-communication library uses its standard network-plugin interface for inter-node data transfers. The OCCL network plugin uses this interface to manage the network paths used for those transfers.

For supported configurations, the OCCL network plugin provides the following capabilities:

  • Cross-plane congestion-aware load sharing: The plugin monitors per-queue-pair (QP) transfer behavior and dynamically assigns collective-communication data to less-congested available paths. This helps distribute traffic across network planes and improve path utilization.
  • Link failure recovery: When it detects a supported link failure, the plugin redirects traffic to a surviving path when one is available. It can automatically resume use of the link after the link is restored.
  • NIC-level failover and recovery: When a supported full NIC failure occurs, the plugin can move collective communication to a surviving NIC and alternate path, and automatically resume use of the restored NIC.
  • Hardware independence: The plugin is designed to operate on OCI-defined supported NICs, switches, and RDMA fabrics without requiring a specific network vendor’s product stack.

The OCCL network plugin supports both single-planar and multi-planar RDMA designs. This gives Oracle a software-based way to improve collective-communication resilience and traffic distribution while preserving flexibility in the underlying network hardware.

Figure 1 below shows AI frameworks using NCCL or RCCL with the OCCL network plugin. The OCCL network plugin manages inter-node collective traffic across available RDMA network planes and paths, without requiring any changes to the AI training framework.

The Oracle OCCL Network Plugin Insertion Point

Figure 1. The OCCL network plugin insertion point

Benchmark Results

In benchmark testing performed in September 2026 on NVIDIA GB300 systems, the OCCL network plugin demonstrated higher average NCCL bus bandwidth in the tested configurations than the same hosts using the same network configuration without the plugin. The tests used 16, 32, 64, and 128 nodes with four GPUs per node; message sizes from 1 GiB through 16 GiB; and 50 measured iterations for each message size. Results shown in the table below are the average of the out-of-place and in-place nccl-tests bus-bandwidth values.

CollectiveNodesWith the OCCL network PluginWithout the OCCL network PluginDifference
AllReduce16359.781 GB/s352.743 GB/s+2.00%
AllReduce32350.587 GB/s331.980 GB/s+5.60%
AllReduce64326.313 GB/s311.884 GB/s+4.63%
AllReduce128218.449 GB/s198.400 GB/s+10.11%
AllToAll1693.171 GB/s84.776 GB/s+9.90%
AllToAll3288.108 GB/s77.724 GB/s+13.36%
AllToAll6483.402 GB/s72.810 GB/s+14.55%
AllToAll12868.147 GB/s63.491 GB/s+7.33%

Note that these results are specific to the tested hardware, software, network configuration, message sizes, and channel settings. The results illustrate the observed performance differences associated with the OCCL network plugin under the tested conditions. They do not represent a performance commitment for all workloads. Actual performance may vary depending on workload and configuration. Customers should follow the official deployment guidance for supported configurations and tuning.

Designed for Existing AI Training Environments

The OCCL network plugin is intended to fit into existing distributed-training environments. For supported configurations, customers do not need to modify their training applications, AI frameworks, or NCCL/RCCL collective calls. They also do not need to deploy a separate control plane, agent fleet, or application-side service.

OCI provides the plugin in two ways:

(1) Oracle HPC images

The plugin is preinstalled in the supported versions of Oracle HPC images. Customers can enable the plugin with the following two CLI commands on the host launched with such an Oracle HPC image:

  • export NCCL_NET_PLUGIN=/opt/occl-net/lib/liboccl-net-occl-ib.so
  • export NCCL_NET=IB

Below is a sample screen capture on a host terminal:

The screenshot example for enabling the OCI OCCL-NET plugin

Customers can use the ls -l /opt/occl-net/lib/ command to get the plugin version on a host console. Below is a sample output capture:

The screenshot example for the outputs of listing the OCI OCCL-NET plugin version

(2) Open-source distribution

Customers using custom images can download the plugin from the OCI GitHub repository at https://github.com/oracle-samples/occl-net and integrate it into their environment.

For more information and user guide, refer to the OCI technical document at https://docs.oracle.com/en-us/iaas/Content/Compute/References/hpc-occl-plugin.htm.

Availability and next steps

The OCCL network plugin is generally available at no additional charge for customers using supported OCI GPU and RDMA-enabled AI infrastructure. Customers continue to pay the standard OCI charges for GPU compute, networking, storage, and applicable support services. Customers can use Oracle HPC images for the preinstalled experience or use the open-source distribution for supported custom images.

To prepare for enabling the OCCL-net plugin on your GPU hosts, review the supported configuration matrix and deployment guide. For more information, see: