Powering interactive 3D experiences on OCI using pixel streaming

April 6, 2022 | 7 minute read
Clarence Ondieki
Cloud Architect
Text Size 100%:

Immersive and real-time interactive applications are undergoing a revolution. From the metaverse to augmented and virtual reality (AR and VR) and gaming applications, distributed 3D content is expected to boom in the coming years. As these experiences and the applications that deliver them get larger and more complex, the capabilities are limited by the capabilities of the devices themselves.

Until recently, users have been limited by the quality of their device’s compute power, display and graphics support, disk space, download speeds, and more. The demand for more power has application developers increasingly relying on the cloud to improve the distribution of 3D content for this new wave of immersive experiences.

A GPU is a specialized processing unit that is optimized for graphical computations and parallel processing, accelerating the creation, and rendering of images, video, and animations. Running 3D applications and interactive experiences in the cloud removes the dependency from the user’s end device and offloads it to the cloud. In the cloud, these applications harness the power of cutting-edge GPUs both for virtual immersion and large-scale data processing. 

Enter OCI

Oracle’s partnership with NVIDIA allows Oracle Cloud Infrastructure to offer GPU-enabled Compute shapes in both virtual machine (VM) and bare metal options, providing flexibility and performance using the latest NVIDIA hardware.

A few benefits of running these applications in OCI include:

  • A global footprint: OCI offers complete cloud capabilities in commercial regions, government regions and on-premises as Cloud@Customer regions. So, you get the flexibility to provide these next-gen experiences with the highest throughput and lowest latency.

  • Performance at every layer of infrastructure: OCI offers an industry-leading flat, non-blocking network, and a non-oversubscribed infrastructure using out-of-the-box virtualization that reduces jitter, ensures a higher packet processing, and ensures that your resources have 100% resource utilization.

  • Optimized costs: OCI gives you the best value in the public cloud with predictable costs as you scale.

  • Security: Move security to the edge with granular controls on your network with a commitment to your compliance requirements.

Unreal pixel streaming

One of the common technologies powering the virtual showrooms, games, and even the metaverse is pixel streaming. Pixel streaming allows developers to stream rendered frames and audio from a remote GPU-enabled computer in the cloud to end users without requiring end users to have GPU hardware. With the ability to enable real-time distributed content to any device, pixel streaming excels in removing the heavy lifting from the client to the server.

Epic Games’ Unreal Engine uses this to stream content to browsers, tablets, smartphones, or any other devices. According to an article by Epic Games, pixel streaming addresses all the user and technical considerations for distributing content in the Age of Experience, where the focus is on deployment of the best content at the highest fidelity available. Pixel streaming can share high-end experiences without consideration of the end client or platform and works within larger platform setups that have various channels for both offline or online content.

Pixel streaming includes the following key components:

  • Signaling server: The signaling server is a web app that establishes a connection and helps the user’s browser communicate with the 3D pixel streaming application.

  • Matchmaker server: This server tracks which signaling servers are available and if they’re being used by client connections. Instead of each client needing to connect to its own signaling and web server URL, they first connect to the matchmaker server. The matchmaker redirects each requester to its own signaling and web server, which then sets up the peer-to-peer connection between the client and its UE4 application. If that connection is active, the matchmaker server doesn’t redirect any new incoming browser connections to the same signaling and web server.

  • Servers running the deployed application

Combining GPUs on OCI with Pixel Streaming, you can interact with 3D applications deployed in the cloud using their normal mouse and keyboard. An Unreal developer logs on to a GPU instance to develop and deploy an Unreal application. The end users then connect and interact with the application when ready. The following graphic shows the high-level architecture of the process:


Figure 1: High-level pixel streaming architecture

Deploying Unreal pixel streaming on OCI

For the simplest scenario, consider a use case where you need multiple users to connect to a single stream with the same session and share viewpoints and controls. Think of a virtual art room where multiple users can all see and interact the same view but can’t manipulate any components in the art room. You can accomplish this scenario using a single VM running in OCI, running both the signaling server and the application, as Figure 2 shows.

For a more complex deployment, think of the metaverse where each user needs to control their avatar in some way, requiring a separate session with a matchmaking server involved. To set up an Unreal pixel streaming application in OCI in an autoscaling manner with matchmaker, using Oracle’s Container Engine for Kubernetes (OKE), you can use this Quick Start can be used, as shown in Figure 3.


Figure 2: Deployment in a single VM


Figure 3: Deployment on Oracle’s Container Engine for Kubernetes

Deployment in a single VM

First, you need to meet the following prerequisites:

Then proceed with the following steps:

  1. Launch an NVIDIA GPU Cloud instance on OCI.

  2. Connect to your instance using remote desktop protocol (RDP) or other protocol.

  3. When connected to the instance, follow this tutorial to install, set up, and connect to a quick sample application.

The following graphics show an example pixel streaming application running on an OCI GPU VM.