Announcing general availability for OCI Compute bare metal instances powered by NVIDIA H100 GPUs

September 19, 2023 | 3 minute read
Sagar Rawal
Vice President of Compute Product Management
Text Size 100%:

Additional contributors: Sanjay Pillai and Andrew Butterfield, OCI Product Management; Leo Leung and Akshai Parthasarathy, OCI GTM

Today, Oracle is announcing general availability of Oracle Cloud Infrastructure (OCI) bare metal Compute instances powered by NVIDIA H100 Tensor Core GPUs and NVIDIA networking.

Improved performance for Compute instances

OCI bare metal instances provide customers with consistent performance, ultra-low latency, isolation, and control. They support applications that require high core counts, large amounts of memory, and high bandwidth. With high-performance local storage, high-performance computing (HPC) storage, cluster networking and memory, bare metal instances can be part of an OCI Supercluster that can scale to tens of thousands of NVIDIA H100 GPUs.

Each of the new BM.GPU.H100.8 instances has eight NVIDIA H100 GPUs. Compared to the previous-generation NVIDIA A100 Tensor Core GPU, the NVIDIA H100 GPU provides up to nine-times faster AI training, thirty-times higher acceleration for AI inferencing, and seven-times higher performance for HPC applications.

We have paired this NVIDIA H100 GPU-enabled server with two Intel Sapphire Rapids CPUs. This gives the server a total of 112 high frequency CPU cores with a base clock of 2.0 GHz and a max turbo clock of 3.8 GHz to efficiently handle the preprocessing of AI training and inference workloads. The Intel Sapphire Rapids offers 4x the PCIE bandwidth and twice the number of PCIE lanes that were built into our previous generation. Time to market was of utmost importance to us. Intel’s Sapphire Rapids CPU aligned best with our development schedule and is a highly performant to pair with the NVIDIA H100 GPU.

Specifications: OCI Compute BM.GPU.H100.8 instance

  CPU   4th Gen Intel Xeon processors, 2x 56 Cores
  GPU   8x NVIDIA H100 80GB Tensor Core
  Memory   2 TB DDR5
  Local storage   16x 3.84 TB NVMe
  Cluster networking     8x 400 Gb/sec

 

Because of its high-performance compute and fast cluster network that “just works,” OCI has been validated by several AI startups and larger organizations as the best cloud platform for building next-generation AI models for both general-purpose and domain-specific AI use cases. As an example, the following customers use OCI AI infrastructure:

  • Adept is building a powerful AI teammate to help with tasks on the computer or internet. OCI Supercluster powered by NVIDIA GPUs has enabled the startup to rapidly advance its general intelligence roadmap and develop its first product: A rich language interface for tools knowledge workers use every day.
  • MosaicML enables enterprises to develop large AI models on their own proprietary data in their secure environments. OCI’s world class infrastructure combined with the MosaicML platform makes large scale generative AI accessible and possible for every industry.

Want to know more?

Learn more about Oracle’s capabilities and start a conversation with us about AI infrastructure today.

Sagar Rawal

Vice President of Compute Product Management

Sagar leads compute product management for OCI and is focused on delivering flexible, scalable, and highly performant infrastructure.  Prior to OCI, he was part of leading compute product at NVIDIA and AWS.


Previous Post

Oracle expands cloud services with new open source data management solutions

Carter Shanklin | 4 min read

Next Post


Hello world! OCI Generative AI is here!

Luis Cabrera-Cordon | 4 min read