Wallaroo helps teams easily operationalize AI models to any cloud, edge or on-premises environment. Providing intuitive capabilities for AI teams to deploy, manage and monitor models in production at scale with ease and repeatability.

A leading player in the AI inference space, Wallaroo has partnered with Oracle Cloud Marketplace to provide a seamless experience for organizations leveraging Oracle Cloud Infrastructure (OCI). As an Oracle Partner, Wallaroo.AI’s integration with the OCI stack represents a significant step forward for enterprises looking to enhance their AI capabilities effectively and efficiently.

The Wallaroo universal AI inference platform enables realtime and batch AI inference on any hardware type (CPU, GPU), and across various AI applications (Time Series, Computer Vision, Classification, Regression and LLMs) in the cloud and at the edge.

 

Deploy, Serve, Observe, Optimize, and Automate AI inference Workloads with Wallaroo at Scale

 

Wallaroo enables enterprises to operationalize AI at scale on any hardware in the cloud, on prem and at the edge. The Wallaroo platform is comprised of 3 core components:

1. Wallaroo Integration Toolkit

 

Wallaroo’s integration toolkit equips AI teams with everything they need to transition from prototype to production seamlessly and intuitively. Integrating with your existing AI toolchain it offers Notebook Integration with platforms like OCI, Databricks, Jupyter, AWS, and Azure ML, and includes built-in data connectors for all major cloud providers.

The toolkit also integrates with both standard and custom orchestration stacks, such as Kubernetes. With ML and LLM Ops APIs, Wallaroo supports continuous integration, deployment, and optimization. Additionally, it provides Inference Logs API and SDK for business reporting integration, Inference Serving APIs for business applications, and custom model registry integration, including native MLFlow Model and Container Registry integration.

2. Wallaroo Inference Engine



 

The Wallaroo Inference Engine built in Rust, is optimized for high-performance in both model and data processing, supporting both batch and real-time workflows. It offers flexible deployment, scaling effortlessly from Raspberry Pi to large Kubernetes clusters, and easily transitions between Ampere, X64, and GPU architectures with simple configuration.

The Wallaroo Inference Engine supports a range of machine learning and deep learning runtimes, including TensorFlow, ONNX, PyTorch, HuggingFace, scikit-learn, XGBoost, and Python. Designed for high availability, it includes compute autoscaling and advanced memory management, along with robust orchestration for complex, multi-step machine learning pipelines and ensembles.

This results in delivering 3x-13x faster inference speeds, real-time latency as low as 1 microsecond, and 5x more efficient data handling. Leading to a 50%-80% reduction in inference costs.

3. Wallaroo Model Ops Center – Centralized Management & Observability

   

Wallaroo’s model ops center provides AI teams a collaborative space to simplify and automate model packaging for deployment while enabling comprehensive monitoring of all models, deployed pipelines, and Inference Servers. Built in Model Registry with Audit Logs and interactive hosted Jupyter Notebooks provides for ease and simplicity of tracking production models.

The platform also supports ML workload automation, allowing for on-demand and scheduled inference workloads with integrated data and compute orchestration. Additionally, Wallaroo offers AI teams full audit capabilities, model observability and data drift detection, and anomaly alerting. Through support for A/B testing, Shadow deployments, and the inline updates of live models it helps ensure seamless operations and continuous optimization in production.

Why Wallaroo

The Wallaroo AI Inference Platform enables enterprise AI teams to quickly and efficiently operationalize AI at scale through an end to end ML and LLM lifecycle from deployment, and ongoing model management with full governance and observability while realizing optimal performance scaling across Ampere, x64 & GPU architectures in the cloud, on-prem and at the edge.

  • Deploy models from testing to production in minutes.
  • 6x faster deployment with self-service tools.
  • Automated auto-scaling and load balancing.
  • Efficient operations reduce latency and costs.
  • Real-time monitoring with proactive analytics.
  • Continuous tracking with drift detection.
  • Seamless inline model upgrades without downtime.

By eliminating delays, reducing costs, and enhancing operational efficiency, the Wallaroo AI inference platform allows your AI teams to focus on high-value tasks with maximum business impact.

Explore Wallaroo.AI on the OCI Marketplace

For organizations looking to enhance their AI capabilities, Wallaroo.AI on the OCI Marketplace is a powerful solution. The integration not only simplifies the deployment of AI models but also provides the tools needed to manage and optimize these models effectively. By combining the strengths of Wallaroo.AI and Oracle Cloud Infrastructure, businesses can unlock new possibilities in AI, driving innovation and growth.

To learn more about Wallaroo.AI’s offerings and explore their solutions on the OCI Marketplace, visit Wallaroo.AI on OCI Marketplace to get started or contact us to learn more. 

This blog is co-written by Younes Amar, VP of Product, Wallaroo.AI