Single root I/O virtualization (SR-IOV) technology enables virtual machines to achieve low latency and high throughput simultaneously on 1 or more physical links. This technology is ideal for low-latency workloads such as video streaming, real-time applications, and large or clustered databases. Hardware-assisted (SR-IOV) networking uses the VFIO driver framework.

Private Cloud Appliance and Compute Cloud @Customer systems running software version 3.0.2-b1443573 or later support the utilization of SR-IOV to pass a physical NIC through to a virtual instance.

This new feature enables instances to have one or more SR-IOV secondary network interfaces. The physical hardware is enabled bydefault and is available to compute instances.

The performance-oriented architecture of Private Cloud Appliance/Compute Cloud@Customer utilizes 100Gbps links in single or double pair(s) based on the compute node hardware type:

  Hardware Type      Network Card Type     Link count per compute node. 
X9-2 ConnectX-5 2x 100Gbp
E5-2L ConnectX-6 4x 100Gbp
E6-2L ConnectX-7 4x 100Gbp

 

Because of this architectural design, instances linked with SR-IOV/VFIO VNICs must also follow the same pattern. A single SR-IOV/VFIO VNIC will have 2 or 4 virtual function interfaces presented to the guest operating system when attached. As such, a bond interface must be created on top of those pair(s) to allow for high availability in case a link goes down, and performance.

This blog post provides a script to automate creation of the network bond interface(s) taking into account the variability in hardware type and SR-IOV/VFIO type VNIC count.

The script comes in 2 flavors:

  1. Cloud-init user-data compatible
    • The cloud-init compatible flavor will allow the user to input in the user-data section unmodified during instance creation time to automate the creation of the bond interface at boot time while using standard OCI Oracle Linux images.
  2. Standalone
    • The standalone flavor is designed to be invoked directly by the user like any other python script. It will be easier to call from other scripted frameworks.

Using the cloud-init user-data compatible script configure_vfio.sh, an instance can be launched with the following OCI CLI command, specifying a subnet not belonging to a VFIO type VCN: 

oci compute instance launch \
–shape VM.PCAStandard1.1 \
–subnet-id ocid1.subnet…. \
–display-name “SR-IOV instance” \
–availability-domain AD-1 \
–source-details ‘{ “imageId”: “ocid1.image…”, “sourceType”: “image”}’ \
–user-data-file configure_vfio.sh \
–ssh-authorized-keys-file $HOME/.ssh/id_rsa.pub 

Once the instance is running, a secondary VNIC must be attached to a subnet belonging to a VFIO type VCN with this command: 

oci compute instance attach-vnic \
–instance-id ocid1.instance… \
–subnet-id ocid1.subnet… 

 

1. configure_vfio.sh

2. Standalone – configure_vfio.py