As AI and machine learning (ML) adoption expands across industries and use cases, customers need more options to optimize performance and price to train, deploy, and fine-tune AI models. To provide more choice, Oracle Cloud Infrastructure (OCI) is introducing a new compute instance for universal AI/ML and graphics workloads.
OCI Compute bare-metal instance with NVIDIA L40S GPUs: BM.GPU.L40S.4
We’re excited to announce the general availability of OCI Compute bare-metal instances with NVIDIA L40S GPUs, BM.GPU.L40S.4. Unique to GPU instances from OCI, bare metal removes the overhead of hypervisors and allows end users to get the most value from each instance’s performance.
You can use the BM.GPU.L40S.4 instance as a standalone virtual workstation with four NVIDIA L40S GPUs. The OCI Supercluster can also use this new instance type, scaling to up to 3,840 L40S GPUs. OCI Supercluster’s ultra-low-latency networking enables training and inference of large language models (LLMs) at medium-scale. OCI’s cluster network uses RDMA over Converged Ethernet Version 2 (RoCE v2) on top of NVIDIA ConnectX-7 network interface cards (NICs) to support high-throughput and latency-sensitive workloads.
OCI’s strategy of ready-to-go virtualization across its entire fleet enables hardware acceleration and efficient network processing for each instance. BM.GPU.L40S.4 uses NVIDIA BlueField-3 DPUs to accelerate networking, storage, and security workloads. For example, I/O transfers to OCI File Storage’s new high-performance mount targets (HPMTs) and the upcoming fully managed Lustre File service (coming soon) will take advantage of DPU hardware acceleration.
With the upgraded 200-Gbps front-end network, each instance can more efficiently move large datasets for data ingestion/retrieval and between storage and GPUs, reducing bottlenecks during model training and inference. Rapid access to vast volumes of data can enable faster iteration and more efficient scaling. The new instance has the following features:
- Instance name: BM.GPU.L40S.4
- Instance type: Bare-metal
- List price: $3.50 Per GPU/Hour
- GPU: Four NVIDIA L40S GPUs with 48 GB each
- CPU: Two 56-core Intel Sapphire Rapids 8480+
- System memory: 1,024 GB DDR5
- Local storage: Two 3.84-TB NVMe
- Hardware acceleration: Enabled by default on NVIDIA BlueField-3 DPUs
- Front-end network: 200 Gbps
- Cluster network: 800 Gbps
- OCI Supercluster scale: Up to 3,840 NVIDIA L40S GPUs
Use cases for BM.GPU.L40S.4
You can use the new instance for the following use cases and more:
- Training, fine-tuning, and inference: With fourth-generation NVIDIA Tensor Cores, BM.GPU.L40S.4 excels at training and fine-tuning small- to mid-sized LLMs and inference across a wide range of generative AI use cases.
- Simulation and digital twins: Bare-metal compute, immediately available virtualization, and NVIDIA RT Cores also make these instances well-suited for developing and deploying real-time 3D applications, developed on OpenUSD and the NVIDIA Omniverse platform, for photorealistic, physics-based simulations and AI-powered digital twins.
- Video transcoding and image processing: BM.GPU.L40S.4 provides best-in-class graphics and media acceleration with third-generation NVIDIA RT Cores and three dedicated encode and decode engines per GPU.
“OCI Compute with NVIDIA L40S GPUs allows us to train our FloodSENS ML model for accurate global flood detection in a cost-effective manner,” said Guy Schumann, CEO of RSS-Hydro.
“We chose OCI AI infrastructure with bare-metal instances and NVIDIA L40S GPUs for 30% more efficient video encoding,” said Sharon Carmel, CEO of Beam Cloud. “Videos processed with BeamrCloud on OCI will have up to 50% reduced storage and network bandwidth consumption, speeding up file transfers by 2x and increasing productivity for end users. Beamr will provide OCI customers with video AI workflows, preparing them for the future of video.”
Getting started
BM.GPU.L40S.4 instances provide a cost-effective GPU solution without compromising on computational power, helping ensure that enterprises can harness AI’s potential. OCI customers can launch the new instance in the Oracle Cloud Console today. For RDMA clustering available in select regions, contact your Oracle sales team and learn more about Oracle Cloud Infrastructure AI Infrastructure.