As organizations adopt AI-powered features, many are looking for ways to keep inference local for security, compliance, and performance reasons. Oracle GoldenGate now makes this possible by supporting local ONNX models, allowing you to leverage AI capabilities without relying on an external model provider.

Note: Local ONNX model support is available beginning with Oracle GoldenGate 23.26.3.

Why Use a Local ONNX Model?

While cloud-hosted large language models are convenient, they aren’t always the right fit. Running a model locally allows GoldenGate deployments to:

  • Keep sensitive data within your environment
  • Support air-gapped or disconnected deployments
  • Reduce inference latency
  • Eliminate dependency on external AI services
  • Avoid configuring and managing provider credentials

This makes local models an excellent option for enterprise and regulated environments.

Deploying a Local Model

Enabling a local ONNX model is straightforward.

Copy your ONNX model and its corresponding tokenizer into the following directory:

ServiceManager/var/lib/models

For example:

ServiceManager/
└── var/
    └── lib/
        └── models/
            ├── my-model.onnx
            └── tokenizer.json

The tokenizer.json file must reside alongside the .onnx model file.

Automatic Model Discovery

Once the files are in place, GoldenGate automatically discovers the local model.

The Service Manager GUI scans the models directory and recognizes:

  • The .onnx model file
  • The matching tokenizer.json

No manual registration is required.

No AI Provider Configuration

One of the biggest advantages of using a local ONNX model is its simplicity.

Unlike cloud-based AI services, local models do not require an AI provider configuration. There is no need to specify:

  • API endpoints
  • API keys
  • Authentication credentials
  • Provider entries

Simply deploy the model files, and the model is available for use.

Bringing AI Closer to Your Data

By supporting local ONNX models, Oracle GoldenGate gives organizations the flexibility to choose where AI inference happens. Whether you’re operating in a secure enterprise environment, deploying at the edge, or working in an air-gapped network, local models provide a practical way to integrate AI while keeping data under your control.

This capability complements GoldenGate’s growing AI integration by giving customers the freedom to choose between cloud-hosted models and locally deployed ONNX models based on their security, performance, and operational requirements.