In the previous blog on CRUNCH CFD, we compared performance of Combustion Research Unstructured Navier-Stokes Solver with Chemistry (CRUNCH CFD) Solver on AWS c5n.18 large instances with and without Elastic Fabric Adapter (EFA) and Oracle Cloud Infrastructure (OCI) BM.Optimized3.36 shape. We noted that Oracle’s high-performance computing (HPC) cluster shows clear performance advantage over AWS, with a 15-percent improvement in time per iteration.

In this blog, we compared Amazon Web Services (AWS) hpc6a.48xlarge instances with OCI BM.Optimized3.36. Oracle worked with CRAFT Tech to conduct a series of benchmarks of the CRUNCH CFD v3.4 solver, a commercial code developed by CRAFT Tech. The multi-physics framework uses a generalized preconditioned flow solver, applicable for all-speed flows.

A graphic depicting the pump used in the benchmark test.
Figure 1: Case of Q/QD = 1.0/sup>

Benchmark setup

We create a cluster of bare metal HPC (BM.Optimized3.36) instances using a stack in the OCI Resource Manager. This stack uses a Terraform template containing all the parts that turn a collection of instances into a functional HPC cluster. The stack can add network file share (NFS) share volumes, extra block volumes, or another file system, such as the OCI File Storage service and SLURM scheduler.

The benchmark test case is a steady Reynolds-averaged Navier-Strokes (RANS) simulation of a single-stage centrifugal pump. It consists of an unstructured mixed element mesh with 48 million vertices with an impeller rotational speed of 1,600 RPM. CFD code for OCI and AWS was compiled with Intel 2021 and Intel MPI 2021.

We used the following specifications for the instances on OCI and AWS:

 

OCI HPC

AWS HPC

Instance Type

BM.Optimized3.36

hpc6a.48xlarge

CPU

Intel Xeon 6354

AMD EPYC 7R13

Cores Per Node

36

96

Memory

512 GB

384 GiB

Interconnect

100 Gbps RDMA

100 Gbps EFA

Storage

3.84 TB NVMe SSD

Amazon FSx for Lustre

Benchmark test case results

Working Group 2 in the HPC project (FY2012-) conducted by Turbomachinery Society of Japan (TSJ) provided the geometric data of the pump. Mr. Hideyo Negishi, JAXA (Tsukuba) provided the computational grid and solution files for this evaluation. Tim Weathers (CRAFT Tech) ran the benchmark.

A point-line chart comparing the iterations of number of processers over time on OCI and AWS where lower is better.
Figure 2: Time taken in seconds to complete one iteration of CRUNCH CFD v3.4 Solver on OCI and AWS

A point-line chart comparing the speedup of the CRUNCH CFD v3.4 Solver on OCI and AWS.
>Figure 3: Speedup comparing OCI and AWS for CRUNCH CFD v3.4 Solver

A point-line chart comparting the parallel efficiency of OCI and AWS running the CRUNCH CFD v3.4 Solver.
Figure 4: Parallel Efficiency comparing OCI and AWS for CRUNCH CFD v3.4 Solver

Conclusion

In this test, OCI’s BM.Optimized3.36 provides a 32–41% performance uplift compared to AWS for CRUNCH CFD v3.4 Solver across 96–1920 cores. Thus, Oracle’s HPC cluster using Intel’s Ice Lake machines show a clear performance advantage over AWS 3rd generation AMD EPYC processors for CRUNCH CFD.

With CRUNCH CFD, other CFD applications are running faster than ever on OCI. If you want to test these applications yourself, start your 30-day free trial and get access to a wide range of Oracle Cloud Infrastructure services for 30 days.

 

Acknowledgements

  • Geometric data of this pump is provided by Working Group 2 in HPC project (FY2012-) conducted by Turbomachinery Society of Japan (TSJ).
  • Computational grid and solution files for this evaluation is provided by Mr. Hideyo Negishi, JAXA (Tsukuba).
  • Tim Weathers (CRAFT Tech) for running the benchmark.