First Principles: Making Kubernetes serverless with OCI Virtual Nodes

March 20, 2023 | 5 minute read
Pradeep Vincent
Senior Vice President and Chief Technical Architect, OCI
Josh Horwitz
Architect, OCI Containers and Kubernetes
Text Size 100%:

Kubernetes has become the standard for container orchestration, which has reduced operational overhead for managing large clusters of container applications. However, Kubernetes has always had this little secret: The underlying cluster of hosts in the Kubernetes system don’t manage themselves. Kubernetes can simplify DevOps but can’t eliminate the operational overhead. Or could they?

Oracle Cloud Infrastructure (OCI) has launched a new feature as part of OCI Container Engine for Kubernetes (OKE) called virtual nodes to bring a serverless Kubernetes experience to its customers. Virtual nodes take away the operational overhead of infrastructure that customers find in traditional managed nodes.

In the following video, we walk through how Kubernetes pods run on Container Instances, our serverless option for running container, to deliver virtual nodes—the first of their kind, infinitely scalable Kubernetes nodes. Check out how we did it and how you can use virtual nodes to dramatically ease cloud operations.

Virtual nodes

A virtual node is a new type of node that looks and acts like a traditional Kubernetes node within the cluster. However, a virtual node is serverless and fully managed by OCI. Unlike a traditional node that runs pods with a local container runtime, virtual nodes rely on Container Instances for the container runtime. You can find a full rundown of container instances and its architecture in our previous First Principles video blog, Inside Container Instances. While managed nodes in OKE aim to make infrastructure simpler, virtual nodes eliminate the infrastructure. Each container instance maps to a single pod. Figure 1 illustrates a virtual node’s logical components. The video provides a detailed overview of pods and nodes and explains the overview of virtual nodes with further technical details.

A graphic depicting the overview of virtual node usage.
Figure 1: Virtual node overview

Virtual nodes are a fully managed, highly available application that look and act like normal nodes to Kubernetes. Virtual nodes rely on the open source Cloud Native Computing Foundation (CNCF) Virtual Kubelet project to provide a translation layer between OCI Container Instances and Kubernetes. This project makes OCI the first major cloud company to offer a fully managed virtual kubelet product that provides a serverless Kubernetes experience. Figure 2 illustrates the architecture for virtual nodes.

A graphic depicting the architecture a deployment using virtual nodes.
Figure 2: Virtual nodes architecture

A virtual node the following main parts:

  • Pod management: Like managed nodes, virtual nodes are responsible for watching for scheduled pods, managing the lifecycle of pods, and reporting the status back to Kubernetes. A key distinction with this solution and other clouds is that a virtual node can run up to 1,000 pods, while other clouds have a 1:1 ratio of pod to node. This capability allows virtual nodes to scale significantly better for larger workloads.

  • HTTPS kubelet API: This API serves the traffic responsible for fetching pod logs, pod exec, pod attach, gathering pod metrics for autoscaling, and more. The HTTPS API for the virtual node is highly available with both healthy members serving incoming traffic instead of only the leader.

The video discusses the virtual node architecture, key scaling, high availability design tradeoffs, and the approach used for providing a more secure and robust solution for customers.

When to use virtual nodes

You can benefit by pivoting to using a virtual node over managed nodes in standard use cases like restful services, batch workloads, artificial intelligence and machine learning, Big Data, continuous integration and deployment (CI/CD), and more. Virtual nodes shine for ephemeral workloads because you’re only billed for the resources requested on the pod and when the pod is finished, the billing automatically stops. Virtual nodes provide a much stronger isolation than real nodes because the pods on a virtual node are backed by container instances instead of cgroup-based container runtimes like Docker.

Real nodes have the benefit of supporting a more diverse set of infrastructure, like bare metal, RDMA, GPU, and supporting complex use cases like gang scheduling for Big Data jobs.

Although today you must choose between a fully serverless cluster with virtual nodes or a cluster of real nodes, we plan to offer support for clusters comprised of both types for ultimate flexibility.

Conclusion

OCI is the first major cloud provider to offer a fully managed virtual node product with the density of over 1,000 pods to a single node. Virtual nodes eliminate operational overhead of traditional container hosts by shifting to a serverless Kubernetes model that uses a fully managed virtual kubelet and OCI container instances. Virtual nodes use OCI Container Instances to run containers, which provides strong isolation through virtualization. While specialized application use cases require traditional nodes, you can benefit by replacing traditional nodes with virtual nodes for most applications.

Oracle Cloud Infrastructure Engineering handles the most demanding workloads for enterprise customers that have pushed us to think differently about designing our cloud platform. We have more of these engineering deep dives as part of this First Principles series, hosted by Pradeep Vincent and other experienced engineers at Oracle.

For more information, see the following resources:

Pradeep Vincent

Senior Vice President and Chief Technical Architect, OCI

Pradeep Vincent is the Chief Technical Architect and Senior Vice President at Oracle Cloud Infrastructure (OCI). He is a technology and software architect with more than 20 years of experience in tech companies such as Oracle, AWS, and IBM. He has a deep understanding of Cloud Infrastructure, Compute, Storage and Networking. Pradeep has been with Oracle for more than eight years leading a team of architects and software engineers building Oracle’s Public Cloud. He also leads OCI’s Architecture and Engineering Community initiatives. 

Show more

Josh Horwitz

Architect, OCI Containers and Kubernetes

Josh Horwitz is an Architect under the Containers and Kubernetes organization in OCI, which includes services like OKE, Functions, Service Mesh and Container Instances. Josh has been with OCI for 4 years and with Oracle for 8 years. Prior to joining OCI, Josh worked in the Oracle Data Cloud GBU where he focused on Kubernetes/containers, big data applications and low latency, high throughput REST APIs.

Show more

Previous Post

E-Business Suite on OCI File Storage: Deployment and configuration best practices

Vinoth Krishnamurthy | 12 min read

Next Post


OKE Workload Identity: Greater control of access

Rishi Johari | 6 min read
Oracle Chatbot
Disconnected