With Oracle Machine Learning for Python (OML4Py) 2.1.1, migrating your datastores between databases is now simpler thanks to new export and import features. Previously, sharing Python and OML4Py proxy objects was restricted to schemas within a single database instance, and moving datastores required a full schema export and import. The latest OML4Py release lets you transfer individual datastores across databases without moving whole schemas. In addition, OML4Py 2.1.1 brings enhancements to ONNX model deployments – see my blog on these enhancements to find out more.

What’s a Datastore?

When working with Python, objects typically exist only in memory and must be written to files for persistence. OML4Py datastores change this by allowing you to store Python objects directly in the database eliminating the need to manage separate files and manually address backup, recovery, and security. This allows you to quickly load your objects into your Python session or access through embedded Python execution.

In addition, datastores can be used for OML4Py proxy objects. These proxy objects reference database objects including tables, views, and models, allowing you to perform operations in the database rather than in Python memory. Within a schema, the proxy objects remain as references to the underlying objects, making it easy to reload them in future sessions or grant access to other users in the database. When you export a datastore, however, the full content of any referenced tables, views, and models is included in the export. This means the exported file size will match the size of the underlying data. The export process captures all objects in the datastore as a complete unit – individual objects cannot be exported separately. If you need to export only a subset of objects, create a new datastore containing just those objects before exporting.

Using Datastore Export and Import

Two new functions support this capability. In the source database, users can export a named datastore to a zip archive in the data pump directory using oml.export_ds:

import oml
oml.export_ds(file_name="my_datastore.zip", name="my_datastore")

Use oml.import_ds to import the datastore to the target schema or database:

oml.import_ds(file_name="my_datastore.zip")

For example, after creating objects in your development database, you can export just the specific datastore you need, connect to your target environment from your Python session, and import it.

import oml
import pandas as pd
from sklearn import datasets, linear_model

# Create a Python dictionary object
my_dict = {'key': 'value', 'data': [1, 2, 3]}

# Create an OML4Py proxy object
diabetes = datasets.load_diabetes()
x = pd.DataFrame(diabetes.data, columns=diabetes.feature_names)
y = pd.DataFrame(diabetes.target, columns=['disease_progression'])
oml_diabetes = oml.push(pd.concat([x, y], axis=1))

# Create and train an sklearn model
regr = linear_model.LinearRegression()
regr.fit(diabetes.data, diabetes.target)

# Save all three object types to a datastore

oml.ds.save(objs={'my_dict': my_dict,
                  'oml_diabetes': oml_diabetes,
                  'sklearn_model': regr},
            name="my_datastore",
            description="My Python and OML4Py objects",
            overwrite=True)

# Export the datastore to the data pump directory
oml.export_ds(file_name="my_datastore.zip", name="my_datastore")

# Import in the target database
import oml

oml.connect(user='oml_user',
            password='oml_user_password',
            host='myhost',
            port=1521,
            service_name='myservice')

oml.import_ds(file_name="my_datastore.zip")

# Describe the contents of the datastore in the target database
oml.ds.describe("my_datastore")[["object_name","class"]]

The results are:

object_name class
0 my_dict dict
1 oml_diabetes oml.DataFrame
2 sklearn_model LinearRegression

Both the export_ds and import_ds functions run in the OML4Py 2.1.1 client with Oracle AI Database 23.26.1. Note, users must be granted read and write access to the database data pump directory by their administrator to use this feature. The data pump directory is an Oracle database directory object used for import and export operations. Your administrator can grant the necessary privileges with:

GRANT READ, WRITE ON DIRECTORY data_pump_dir TO your_username;

Replace your_username with your database username.

Resources

For more information and to get started with datastore export and import:

OML4Py Downloads
Oracle Machine Learning for Python Documentation
Oracle Machine Learning for Python Datastore Documentation
Oracle Data Pump Documentation for Oracle Autonomous AI Database
Oracle Data Pump Documentation for Oracle AI Database
OML4Py Hands-On Lab