Deliver better price-performance by selecting the right cloud to run APISIX API gateway

November 8, 2022 | 6 minute read
Kailas Jawadekar
Director of Product Marketing
Sagar Zanwar
Principal Product Manager, Compute
Text Size 100%:

This guest blog was written in collaboration with Shirui Zhao from API7.ai.

For many organizations, the flexibility and on-demand nature of public clouds make them an attractive option for their workloads. However, not all public clouds are the same. For organizations moving their workloads to the public cloud, choosing the right cloud service provider (CSP) and compute architecture is critical for finding the right balance between performance and cost.

Arm architecture has become the cornerstone of the world’s largest computing ecosystem, embedded systems, and mobile devices. Many experts regard it as the future of cloud computing due to its advantages of low power consumption, low cost, high performance, and flexible licensing. Now, the technology has evolved to support hyperscaler data centers and cloud computing. Arm-based processors are interesting for cloud customers because they scale linearly, provide predictable performance, and provide the highest density of cores at a lower price point. With Arm-based cloud compute, customers can run existing workloads at a lower cost and build new applications with superior economics and performance.

Arm servers of major cloud vendors

Leading cloud vendors like Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, and Oracle Cloud Infrastructure (OCI) have successively launched Arm-based servers. One of the main advantages of using Arm architecture is cost efficiency because less power is consumed. This blog selects Arm instances from these vendors to conduct performance testing. Let’s first examine the four major manufacturers and their products.

OCI, Azure, and GCP use Ampere Altra CPUs (Arm N1 cores) to provide Arm-based Compute instances. Ampere is offering a 64-bit processor-powered server dedicated to cloud computing. Ampere Altra can deliver higher cores (up to 80) per socker compared to AWS Graviton. Cloud providers can use their data center space more efficiently, improving price-performance while reducing power costs. Ultimately, these service providers can pass on those savings to customers, many of whom are becoming increasingly aware of the environmental impact of their computing activities.

Contrastively, AWS designed Graviton processors with high-performance cores to run cloud workloads in Amazon EC2. Graviton 3 is based on latest Arm V1 cores.

How we tested

We measured the overall performance of each server by testing single-core performance. We selected the network IO-intensive API gateway Apache APISIX to bind a single-CPU core for stress testing on the four models: AWS c7g.large, OCI A1, GCP t2a-standard-2, and Azure D2ps v5. Although the name contains D2ps, it’s a dual-core CPU belonging to the Dpsv5 series. We used the Apache APISIX’s official open-source performance benchmark and stress tested to analyze server performance through two metrics: QPS and response latency.

Apache APISIX is a cloud native, high-performance, scalable, open source API gateway. Compared with traditional API gateways, Apache APISIX is based on NGINX and LuaJIT with features, such as dynamic routing and plug-in hot reloading, which suits API management under cloud native architecture. The following diagram shows the architecture.

 

How We Tested It

 

Test cases

We tested the performance of Apache APISIX in the following two typical scenarios, obtaining more realistic test data for comparison.

In the first scenario, we used a single upstream without any plugins to test the performance of Apache APISIX in pure proxy back-to-origin mode. The second scenario uses a single upstream with two plugins. It mainly tests the performance of APISIX when the two-core consumption performance plug-ins, limit-count, and prometheus are operating.

Spend less by running APISIX API gateway by choosing the right cloud provider and Arm instance

The following table summarizes the cost and price-performance ratio for OCI A1, AWS c7g. large, Azure, and GCP t2a-standard-2 running for one year, referring to the queries-per-second (QPS) data for a single upstream in the Apache APISIX performance test. The larger the number, the higher the QPS that customers can obtain at the unit price. OCI A1 outperforms other CSPs for price-performance, especially compared to newer generation Graviton 3-based c7g instances, which have higher performant cores. This improvement translates into cloud savings for customers and highlights why customers must consider both performance and cost when evaluating cloud services.

Cost-Performance

 

CSPs and their annual cost of running Arm instances and cost-performance ratio

CSP

Annual cost

Cost performance (QPS/cost)

OCI A1 (Ampere)

$280

41

AWS c7g.large

$633

36

Azure D2ps v5

$398 (1 year 41% of)

34

GCP T2a

$675

15

Test results

The following figure shows the queries per second test result of AWS c7g.large, GCP t2a-standard-2, Azure D2ps v5, and OCI A1. The higher the QPS value, the better the performance of the server.

In scenario 1, with a single upstream without any plugins, AWS c7g.large achieves a QPS of 23,000 times/sec, almost twice the performance of GCP t2a-standard-2 (11,300 times/sec QPS). A small gap exists between Azure D2ps v5, OCI A1, and GCP t2a-standard-2. OCI A1 and GCP t2a-standard-2 have almost the same performance with a difference of only 200 times/sec. However, factoring in price, we see that A1 provides consistently better price-performance compared to other CSP’s. OCI A1 performs better with Oracle Linux; we have a test in progress using that parameter.

In scenario 2, with a single upstream and two plug-ins, the QPS of AWS c7g.large reaches 18,000 times/sec, still leading while narrowing the gap with the other servers. The performance of Azure D2ps v5 is slightly higher than that of OCI A1 with a difference of only 400 times/sec.

RPS

 

Conclusion

In addition to the performance aspects, factoring in the cost of running your workloads on various CSPs is critical. Organizations can deliver the real value to their customers while reducing the operating cost at scale.

Through the analysis of the performance test results of Apache APISIX and accounting for yearly costs, Oracle Cloud Infrastructure A1 outperforms other CSPs for price-performance, giving IT organizations the real savings.

For more information on the details in this blog, see the following resources:

Kailas Jawadekar

Director of Product Marketing

Sagar Zanwar

Principal Product Manager, Compute


Previous Post

Training large language models like GPT-3 on Oracle Cloud NVIDIA GPU shapes

Sanjay Basu PhD | 9 min read

Next Post


Terraform guardrails: Getting started with testing your deployments

Gideon Wulfsohn | 3 min read