Kubernetes cost comparison: Who provides the best value?

March 20, 2023 | 16 minute read
Brian Wood
Product Marketing (AI for AppDev)
Text Size 100%:

Kubernetes is an open source container orchestration solution for automating the deployment, scaling, and management of containerized software. Gartner® estimates that, “By 2027, more than 90% of global organizations will be running containerized applications in production, which is a significant increase from fewer than 40% in 2021.”¹

As Kubernetes usage and footprints grow, customers look for ways to rein in costs. Given that Kubernetes is an open source framework, the technology itself and the key features of managed services are largely similar across providers -- but what you pay for these services varies greatly.

In this blog, we walk through how Oracle Cloud Infrastructure (OCI) can save you substantial costs over Amazon Web Services (AWS), Microsoft Azure, and Google Cloud for managed Kubernetes services, even when comparing against their lowest cost regions.

Comparing Kubernetes services

All four hyperscalers offer a managed Kubernetes service, each aiming to help reduce your time, cost, and operational burden of managing the many complexities of Kubernetes infrastructure. There are no minimum fees, no upfront commitments required, and you pay only for the underlying resources and any applicable management fees. All services automatically manage the availability and scalability of the Kubernetes control plane nodes and offer varying levels of control for the worker nodes.

While not identical, each service enables you to configure clusters with two main types of worker nodes:

  • Serverless: The cloud service takes on full management of your worker nodes including configuring, upgrading, scaling, and so on

  • Managed: You can configure and managed the cloud compute instances as needed

More often, customers are choosing the serverless option, offloading ongoing operations to the service in order to focus on value-add priorities instead of managing infrastructure. Here are the respective serverless options for each Kubernetes service:

Now, let’s explore how the respective Kubernetes services are priced and compare two common workload scenarios to see which offers the better economic choice.

Happy colleagues collaborating

Comparing Kubernetes prices

The following table shows the lowest rates published for the two comparison scenarios in this blog post: a serverless (virtual) Kubernetes experience and one with managed nodes, both of which use x86-based processors and Linux images.

Note that although pricing for AWS, Azure, and Google Cloud services varies widely by region, we’re using their lowest published regional rates to show each provider in its best light. Even so, observe how much lower the OCI OKE rates are for nearly every metric; the deltas are dramatic.

Another important distinction between cloud providers are the differences in serverless Kubernetes nodes-to-pods implementation, which impacts costs:

  • Each Amazon EKS pod is provisioned on its own AWS Fargate node. Similarly, each Azure AKS pod is provisioned on its own AKS virtual node.  
  • In contrast, OKE customers can run multiple pods on a single virtual node, which essentially functions as a “bucket of unlimited compute resources” that can be allocated to n-number of pods. Additional nodes are not required in OKE for increasing capacity because scaling and compute/memory charges are done at the pod level rather than at the node level. This means that with OKE, you only need either one or three virtual nodes; typically, one node is used for development purposes and three virtual nodes are recommended for production workloads to ensure high availability.
Lowest published rates for Kubernetes with serverless (virtual) nodes

Metric

Unit

EKS

AKS

GKE

OKE

Region

Name

US East (N. VA)

East US

South Carolina

Any region

Cluster

Cluster per hour

$0.10

$0.10

$0.10

$0.10

Serverless (Virtual) experience

Serverless

Name

Fargate

Virtual Nodes

Autopilot

Virtual Nodes

CPU

vCPU per hour

$0.04048

$0.0405

$0.0445

$0.0125

Memory

GiB per hour

$0.004445

$0.00445

$0.0049225

$0.0015

Virtual node

Node per hour

-

-

-

$0.015

Managed Nodes (16 vCPU, 64 GiB memory)

Node

Name

m6a.4xlarge

D16as v5

e2-standard-16

E4.Flex

On-demand

Node per hour

$0.6912

$0.688

$0.536048

$0.296

Boot volume

GB per month

$0.08

$0.081

$0.10

$0.0425

 

These published rates have the following details:

  • All calculations use prices published as of March 1, 2023. We have attempted to use “apples-to-apples” comparisons as much as possible in terms of processor configurations (e.g., manufacturer, type, generation, speed, memory, bandwidth, and so on).

  • Common cloud industry practice is to define compute instances based on the number of virtual CPUs they include. Each vCPU provides the capacity for one thread of runs. A vCPU does not provide a whole physical compute core but is instead part of a core. In contrast, Oracle’s x86 compute shapes use OCPUs, which equate to physical CPU cores, each of which provides for two threads. So, two vCPUs on AWS, Azure, or Google Cloud compare to 1 OCPU on OCI.

  • In our example calculations below we’re assuming production workloads and thus are using three OKE virtual nodes.

  • While AKS offers a free tier for getting started, they recommend the standard tier for all mission critical, at scale, or production workloads -- hence we are using their standard tier pricing.

  • Google Cloud GKE offers a free tier which provides up to $74.40 in monthly credits, the equivalent of one free cluster per month ($0.10 per hour for 24 hours a day, for 31 days), which we apply in the comparisons.

Caution: AWS, Azure, and Google Cloud prices vary widely by region

Prices for Amazon EKS with AWS Fargate (for serverless) or without Fargate (for managed nodes) vary widely across AWS regions and can be much higher than in the US East region in northern Virginia. The following examples show the higher AWS Fargate CPU costs across different regions when compared to US East:

  • 15% higher in US West, N. California (serverless), 16% higher (managed)

  • 15% higher in Frankfurt (serverless), 19% higher (managed)

  • 25% higher in Tokyo (serverless), 29% higher (managed)

  • 35% higher in Zurich (serverless), 56% higher (managed)

  • 72% higher in São Paulo (serverless), 59% higher (managed)

Likewise, CPU prices for AKS virtual nodes in many other Microsoft Azure regions are substantially higher than in East US:

  • 15% higher in West US (serverless), 17% higher (managed)

  • 25% higher in Japan East (serverless), 30% higher (managed)

  • 27% higher in Switzerland North (serverless), 43% higher (managed)

  • 50% higher in Germany North (serverless), 57% higher (managed)

  • 100% higher in Brazil South (serverless), 60% higher (managed)

Similarly, CPU prices for GKE Autopilot in other Google Cloud regions can be much higher than their lowest rates in South Carolina:

  • 20% higher in Los Angeles (serverless), 20% higher (managed)

  • 28% higher in Tokyo (serverless), 28% higher (managed)

  • 29% higher in Frankfurt (serverless), 29% higher (managed)

  • 40% higher in Zurich (serverless), 40% higher (managed)

  • 59% higher in São Paulo (serverless), 59% higher (managed)

In contrast, OCI offers consistent pricing and availability for all services, including OKE virtual nodes, across all global regions. This predictability makes it easy for you to plan and budget for rapid geographic expansion regardless of where you need to consume the cloud services.

How to calculate Kubernetes cloud costs

Total monthly price = Cluster/Node fee + CPU fee + Memory fee + Boot volume fee

The fees are calculated as follows:

  • Cluster: (# clusters) * (price-per-cluster-hour) * (# daily-cluster-duration-hours) * (# days)

  • Node: (# nodes) * (price-per-node-hour) * (# daily-node-duration-hours) * (# days)

  • CPU: (# nodes) * (# vCPU) * (price-per-vCPU-hour) * (# daily-CPU-duration-hours) * (# days)

  • Memory: (# nodes) * (mem-in-GiB) * (price-per-GiB-hour) * (# daily-mem-duration-hours) * (# days)

  • Boot: (# nodes) * (storage-in-GB) * (price-per-GB-month)

Note that the nominal OKE node fee only applies to serverless (virtual) nodes, and, for all providers, boot volume storage is only charged for managed node scenarios.

Man in front of laptop

Comparing Kubernetes costs for common use cases

To compare the true cost of real-world use cases, we use two scenarios that represent common workloads: a typical small serverless Kubernetes use case and then a moderate size managed node use case.

Scenario A: Kubernetes with serverless (virtual) nodes

In this use case, we have a single serverless cluster of 20 pods running every day for 31 days, each configured with 2 vCPU and 8 GiB memory.

EKS with Fargate

  • Cluster: (1 cluster) * ($0.10/cluster-hour) * (24 hours) * (31 days) = $74.40

  • Node: no charge

  • CPU: (20 pods) * (1 node/pod) * (2 vCPU/node) * ($0.04048/vCPU-hour) * (24 hours) * (31 days) = $1,204.68

  • Memory: (20 pods) * (1 node/pod) * (8 GiB/node) * ($0.004445/GiB-hour) * (24 hours) * (31 days) = $529.13

  • Total: $74.40 + $1,204.68 + $529.13 = $1,808.21 for EKS with Fargate

AKS Virtual Nodes

  • Cluster: (1 cluster) * ($0.10/cluster-hour) * (24 hours) * (31 days) = $74.40

  • Node: no charge

  • CPU: (20 pods) * (1 node/pod) * (2 vCPU/node) * ($0.04050/vCPU-hour) * (24 hours) * (31 days) = $1,205.28

  • Memory: (20 pods) * (1 node/pod) * (8 GiB/node) * ($0.00445/GiB-hour) * (24 hours) * (31 days) = $529.73

  • Total: $74.40 + $1,205.28 + $529.73 = $1,809.41 for AKS Virtual Nodes

GKE Autopilot

  • Cluster: free since the GKE free tier provides 1 free cluster per month

  • Node: no charge

  • CPU: (20 pods) * (2 vCPU/pod) * ($0.0445/vCPU-hour) * (24 hours) * (31 days) = $1,324.32

  • Memory: (20 pods) * (8 GiB/pod) * ($0.0049225/GiB-hour) * (24 hours) * (31 days) = $585.97

  • Total: $1,324.32 + $585.97 = $1,910.29 for GKE Autopilot

OKE Virtual Nodes

  • Cluster: (1 cluster) * ($0.10/cluster-hour) * (24 hours) * (31 days) = $74.40

  • Node: (3 virtual nodes) * ($0.015/virtual-node-hour) * (24 hours) * (31 days) = $33.48

  • CPU: (20 pods) * (2 vCPU/pod) * ($0.0125/vCPU-hour) * (24 hours) * (31 days) = $372.00

  • Memory: (20 pods) * (8 GiB/pod) * ($0.0015/GiB-hour) * (24 hours) * (31 days) = $178.56

  • Total: $74.40 + $33.48 + $372.00 + $178.56 = $658.44 for OKE Virtual Nodes

AWS, Azure, and Google Cloud are each nearly three times as expensive as OCI for Scenario A, even with the lowest cost region for these other services.

Scenario A: Kubernetes cloud cost comparison with serverless (virtual) nodes

EKS with Fargate

AKS Virtual Nodes

GKE Autopilot

OKE Virtual Nodes

$1,808.21

$1,809.41

$1,910.29

$658.44

2.7 times higher than OCI

2.7 times higher than OCI

2.9 times higher than OCI

OCI has lowest cost and best value

Hands coding on laptop

Comparing Kubernetes costs with managed nodes

Now let’s turn our attention to a scenario using managed nodes. We already gathered the comparable CPU, memory, and storage pricing for each provider to meet the scenario’s requirements in the pricing table above.

Scenario B: Kubernetes with managed nodes

In this use case, we have a single cluster of 20 pods running every day for 31 days, each with 16 vCPU, 64 GiB memory, and 50 GB boot storage volumes.

EKS using m6a.4xlarge

  • Cluster: (1 cluster) * ($0.10/cluster-hour) * (24 hours) * (31 days) = $74.40

  • Node: (20 pods) * (1 node/pod) * ($0.6912/node-hour) * (24 hours) * (31 days) = $10,285.06

  • Boot: (20 pods) * (1 node/pod) * ($0.08/GB-month) * (50 GB) = $80.00

  • Total: $74.40 + $10,285.06 + $80.00 = $10,439.46 for EKS

AKS using d16as v5

  • Cluster: (1 cluster) * ($0.10/cluster-hour) * (24 hours) * (31 days) = $74.40

  • Node: (20 pods) * (1 node/pod) * ($0.688/node-hour) * (24 hours) * (31 days) = $10,237.44

  • Boot: (20 pods) * (1 node/pod) * ($0.081/GB-month) * (50 GB) = $81.00

  • Total: $74.40 + $10,237.44 + $81.00 = $10,392.84 for AKS

GKE using e2-standard-16

  • Cluster: free since the GKE free tier provides 1 free cluster per month

  • Node: (20 pods) * (1 node/pod) * ($0.536048/node-hour) * (24 hours) * (31 days) = $7,976.39

  • Boot: (20 pods) * (1 node/pod) * ($0.04/GB-month) * (50 GB) = $100.00

  • Total: $7,976.39 + $100 = $8,076.39 for GKE

OKE using E4.Flex

  • Cluster: (1 cluster) * ($0.10/cluster-hour) * (24 hours) * (31 days) = $74.40

  • CPU: (20 pods) * (1 node/pod) * (16 vCPU) * ($0.0125/vCPU-hour) * (24 hours) * (31 days) = $2,976.00

  • Memory: (20 pods) * (1 node/pod) * (64 GiB) * ($0.0015/GiB-hour) * (24 hours) * (31 days) = $1,428.48

  • Boot: (20 pods) * (1 node/pod) * ($0.0425/GB-month) * (50 GB) = $42.50

  • Total: $74.40 + $2,976.00 + $1,428.48 + $42.50 = $4,521.38 for OKE

AWS, Azure, and Google Cloud are about twice as expensive than OCI for Scenario B, even with the lowest cost region for these other services.

Scenario B: Kubernetes cloud cost comparison with managed nodes

EKS

AKS

GKE

OKE

$10,439.46

$10,392.84

$8,076.39

$4,521.38

2.3 times higher than OCI

2.3 times higher than OCI

1.8 times higher than OCI

OCI has lowest cost and best value

Another option for managed nodes: Preemptible instances

Preemptible instance support for OKE managed nodes is another option for saving money. Also called spot instances (or spot virtual machines) by other cloud providers, preemptible instances can further reduce your OKE spend as they are priced at a 50% discount when compared to regular on-demand OCI instances.

OCI preemptible instances don’t guarantee capacity or performance and are only suitable if your workload can tolerate interruptions or flexible start and stop times. You can delete preemptible instances at any time with minimal notice, so they’re not suitable for long running workloads but are great for ephemeral use cases.

Conclusion: OCI delivers the best value for Kubernetes

The evidence is clear: Even in competitors’ lowest-cost regions, Oracle Container Engine for Kubernetes (OKE) provides a substantial cost advantage over AWS, Microsoft Azure, and Google Cloud for typical use cases. As the comparison scenarios in this blog post show, in most cases OKE is less than half the cost of the competition.

These cost differences can really add up, particularly when workloads scale. For example, using published list pricing, one prominent OCI financial services customer would spend over $723K monthly if they were to use Amazon EKS with AWS Fargate, yet only about $335K per month using OKE (i.e., less than half the cost). They could use that massive savings -- nearly $400K per month, or over $4.6M annually -- to fund many other of the company’s business imperatives.

The cost differences are even greater in many other public cloud regions globally where AWS, Azure, and Google Cloud have CPU prices much higher than their lowest rates -- up to 100% more in some cases. All this information is critical as you look to enable Kubernetes at ever-growing scale while saving on your overall total cost of ownership.

Learn more about OKE and try the service for yourself, read more about the latest set of OKE capabilities to simplify large-scale Kubernetes environments at reduced costs, or use the OCI Cost Estimator to price out your own workload scenarios. You can also contact us to speak with a cloud architect to discover how OKE can help you accelerate and simplify adoption of Kubernetes and scale its usage across your organization in a cost-effective way.

 

¹ Gartner, CTOs’ Guide to Containers and Kubernetes — Answering the Top 10 FAQs, May 2022. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Brian Wood

Product Marketing (AI for AppDev)


Previous Post

OKE virtual nodes deliver a serverless Kubernetes experience

Greg Verstraeten | 7 min read

Next Post


Kubernetes at scale just got easier with new Oracle Container Engine for Kubernetes enhancements

Mickey Boxell | 6 min read