News, tips, partners, and perspectives for the Oracle Linux operating system and upstream Linux kernel work

An Update on Retpoline-enabled Kernels for Oracle Linux

In January, researchers disclosed flaws in speculative execution known as Meltdown and Spectre. Oracle published official guidance in this support note: Responding to the Spectre and Meltdown vulnerabilities (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754) in Oracle Linux and Oracle VM on Oracle X86 Servers (Doc ID 2370398.1)

At that time, we shipped mitigations for these security issues which depended on special Intel microcode. We're excited to announce that our latest kernel release kernel-uek-4.1.12-112.16.4 contains faster, retpoline-based mitigations for Spectre Variant 2 (CVE-2017-5715). This kernel is available for both Oracle Linux 7 and Oracle Linux 6. Along with the existing patches for UEK Release 2, UEK Release 3 and the Red Hat Compatible Kernel for Oracle Linux 6 and 7, this provides a full complement of the latest software mitigations for the Spectre and Meltdown vulnerabilities in Oracle Linux. And we've published all the source on github.com here.

Oracle also ported the Spectre Variant 2 mitigations into Xen, using IBRS/IBPB mitigations in January. We're about to release retpoline mitigations for Xen on Oracle VM 3.4. This will ensure full protection from Meltdown/Spectre-type attacks for all our supported hypervisors: Oracle VM 3.2, Oracle VM 3.3 and Oracle VM 3.4 and kvm.

A discussion of the advantages of Retpolines can be found in this intel.com white paper and in this google.com support article. Retpolines are a software mitigation performed by the compiler which isolates indirect branches from speculative execution. Derived of "return trampoline", retpoline mitigations have significantly less performance overhead than microcode-based mitigation, and under some workloads can bring performance to near pre-patched levels.

Retpolines are enabled by recompiling the kernel (and kernel modules) with a retpoline-aware gcc compiler, which is available in Oracle Linux 7 (and also Oracle Linux 6). Our compiler experts ported this support into the gcc-4.8 and gcc-4.4 compilers, and the compilers are available on yum.oracle.com for public download. This was a prerequisite to making retpoline-enabled kernels available on Oracle Linux, which could use the compiler features to self-protect the kernel against the Spectre Variant 2 attacks. Recompilation of applications is not required.

The alternative to using retpoline is IBRS, Indirect Branch Restricted Speculation, and calls a special SPEC_CTRL MSR (model-specific register) defined in the latest microcode updates from Intel. IBRS uses microcode to mitigate the security vulnerabilities. IBRS causes a significant drop in performance under some workloads. A second MSR, IBPB (Indirect Branch Predictor Barrier) is still used for some specific use cases even when retpoline is available.

There are a number of caveats to using retpolines as a mitigation: first, the hardware has to support retpoline: some modern hardware can ignore the retpoline mitigation and continue speculating instructions. Second, any loadable kernel modules must also be compiled with a retpoline-aware compiler, otherwise the kernel can still be vulnerable.

The latest kernel-uek will automatically detect each of these conditions and enable microcode-based IBRS mitigation. The fallback, IBRS mitigation, requires updated microcode on the system. Therefore we always recommend updating system microcode to the latest-available from your hardware vendor. The updated Intel microcode introduces the SPEC_CTRL MSR but does not invoke it -- the kernel has to invoke the MSR. This kernel behavior can be enabled or disabled by the user, so loading the updated microcode on a system where you plan to disable IBRS will not have a performance impact.

Microcode does not need to be updated in guest (virtual machine) systems: as long as the host system has the correct microcode and updated software (Xen or qemu), the hypervisor will pass through the MSRs necessary for the guest to protect itself.

Third Party Kernel Modules: Any third-party kernel modules must be recompiled with a retpoline-aware compiler. While the kABI guarantees in UEK ensure that previously-compiled modules will load, if those modules are not retpoline aware then the whole kernel will re-enable IBRS protections and the performance advantage of retpolines will be lost. 

This includes customers running Oracle Grid Infrastructure software: please update your kernel modules to retpoline-compiled versions! There's a tracking bug (Bug 27463879) for recompiling usm drivers with latest gcc and several MOS notes to help you with this process: ACFS -  MOS Note 1369107.1 and ASMFD - MOS Note 203468.1 as well as specific notes for Exadata (Note 2356385.1) and ODA (Note 2377658.1).

Repolines are not required. Retpoline-enabled kernels provide a performance enhancement, but if you have a security-patched kernel without retpolines, it is not critical to pull in these patches immediately.

Microcode updates are required: Many scenarios exist where the system may have to fall back to IBRS (microcode-based) mitigations, which will fail if the microcode has not been updated on the system. It's imperative, even if you are able to take advantage of retpolines, to have the microcode available as a fallback.  There are numerous edge cases (kvm, hardened GPG, Xen, hardware limitations, ..) where retpoline mitigations are not sufficient. You don't want to see the following message, which will appear in your 'dmesg' output if the microcode is out-of-date.

[  358.742211] kmod: loading module not compiled with retpoline compiler.
[  358.742214] Spectre V2 : Disabling Spectre v2 mitigation retpoline.
[  358.749417] Spectre V2 : Could not enable IBRS.
[  358.754569] Spectre V2 : No Spectre v2 mitigation to fall back to.
[  358.761587] Spectre V2 : system may be vulnerable to spectre

Boot-time logs if retpolines are not possible and IBRS-capable microcode is not available.

No application recompile: There is no need to recompile applications to allow the kernel to use retpoline; only loadable kernel modules must be recompiled.

In summary:

  • Oracle Linux 6 and 7  with UEK 2, 3 and 4 as well as RHCK address Spectre variants 1, 2, and 3
  • Our latest Unbreakable Enterprise Kernel release 4 include retpoline-based mitigations for Spectre variant 2
  • The retpoline-enabled UEK4 offers a significant performance boost over previous releases with microcode-basd Spectre mitigations
  • UEK4 with retpoline mitigations will only work on certain hardware and requires all kernel modules to be compiled with a retpoline-aware compiler
  • UEK4 with retpoline mitigations will fall back on microcode-based protections automatically if conditions necessary for retpoline support are not met
  • All this and more in the My Oracle Support document: Doc ID 2370398.1

Join the discussion

Comments ( 2 )
  • Gadi Friedman Sunday, August 26, 2018
    In the summary section you wrote:
    "Oracle Linux 6 and 7 with UEK 2, 3 and 4 as well as RHCK address Spectre variants 1, 2, and 3"
    - It seems that SpectreV2 is solved on all UEKs 2,3, and 4.

    But in the next line you wrote
    "Our latest Unbreakable Enterprise Kernel release 4 include retpoline-based mitigations for Spectre variant 2"
    - It seems that only UEK contains that fix.

    What is the right one?
  • Sergio Wednesday, September 12, 2018
    @Gadi - both statements are correct. All the UEK releases Greg lists address Spectre v2. Only UEK4 addresses it via retpolines.
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.