Scaling Production AI with WEKA on Oracle Marketplace

The economics of production AI depend on how much useful work an organization gets from its infrastructure. For training teams, that means keeping GPUs supplied with datasets and checkpoints. For teams serving AI applications, it means generating more tokens for more users without repeating computation or continually expanding the GPU fleet. Storage and memory constraints can limit both outcomes, even when the underlying compute capacity is available.

On Oracle Cloud Infrastructure (OCI), WEKA’s NeuralMesh offerings support AI workloads from training through inference. NeuralMesh Axon places storage close to GPU compute to help training teams access datasets and checkpoints, while NeuralMesh with Augmented Memory Grid helps inference teams retain and reuse context as demand grows. These capabilities can run within a customer’s OCI tenancy for greater control over the environment. Oracle and WEKA are also planning a more integrated Oracle Marketplace experience to simplify purchasing and billing.

What the Planned Marketplace Experience Changes

Oracle and WEKA are enhancing the purchasing experience for WEKA’s NeuralMesh offerings on Oracle Cloud Infrastructure (OCI). NeuralMesh Axon supports AI training by placing high-performance storage close to GPUs, while NeuralMesh with Augmented Memory Grid helps inference workloads retain and reuse context. This Marketplace launch aims to simplify purchasing and billing for Oracle customers. 

Oracle customers can now apply eligible Oracle Universal Credits towards WEKA offerings on OCI, maximizing their existing cloud investments and enabling robust production AI. 

The integration provides customers with a clearer path from offer to billing for WEKA solutions deployed on OCI. With this streamlined procurement process, the Oracle and WEKA teams can help customers select a deployment and offer that fully meets their AI training and inference requirements.

From Training Data to Inference Context on OCI

NeuralMesh by WEKA provides high-performance storage on OCI for AI training and inference. Training pipelines need to move datasets into GPU clusters and write checkpoints while handling both large transfers and millions of small files. NeuralMesh Axon runs directly on OCI GPU instances and uses local NVMe capacity, placing storage close to compute rather than in a separate silo.

The Center for AI Safety (CAIS) provides a customer example. Storage bottlenecks and slow metadata lookups constrained its growing research cluster, while dataset copies added to costs. CAIS deployed NeuralMesh Axon using NVMe capacity already in its OCI compute environment. In WEKA’s case study, CAIS reported a 90% reduction in cloud storage costs and an estimated fivefold improvement in storage performance over its prior solution. Its research community grew from about 30 to more than 200 active researchers. These are results from CAIS’s deployment, not performance guarantees for other workloads.

Inference presents a different challenge. Applications with long conversations or agentic workflows retain prior context in a key-value (KV) cache so they can reuse previous computation. When that cache exceeds available memory, useful context may be evicted and recomputed, limiting throughput as demand grows. NeuralMesh with Augmented Memory Grid on OCI addresses this constraint with a persistent, shared cache backed by NeuralMesh-managed NVMe storage.

Joint OCI and WEKA testing provides a compelling inference proof point. On a nine-node OCI cluster with 72 H100 GPUs and 100,000-token inputs, Augmented Memory Grid supported more than 5,000 concurrent users, compared with about 600 for a DRAM-only configuration on the same GPU footprint. With Augmented Memory Grid, customers can support 10x more simultaneous AI users and deliver 10x higher token throughput without increasing their GPU footprint. That’s 10x the work of a GPU for the same cost.

The CAIS deployment and OCI benchmark show how different infrastructure constraints affect AI economics. For training, faster data access and better use of local storage can help keep GPUs productive. For inference, retaining reusable context can help an existing cluster serve more users and generate more tokens. Customers should evaluate each result against their own workloads and cost measures.

How Customers can get started

Customers can explore NeuralMesh by WEKA and NeuralMesh with Augmented Memory Grid on Oracle Marketplace. Teams focused on training data, checkpointing, or storage efficiency can evaluate NeuralMesh Axon against their data access needs. Teams building long-context applications can assess KV-cache behavior, concurrency targets, token throughput, and the published OCI inference results when evaluating Augmented Memory Grid.

To get started, discuss your workload with an Oracle account team or WEKA. Share the size and shape of your data, GPU environment, expected growth, and measures of success. For inference, those measures may include concurrent users, tokens delivered per GPU, and cost per token. This information can guide architecture and sizing and help you assess the published results against your own needs. You can also explore the dedicated WEKA Marketplace destination and ask about a tailored Private Offer.

Learn more

For more information, check out the Oracle Marketplace.