Today, an organization’s data assets are typically spread across multiple operational Oracle databases on premises or in the cloud, Oracle AI Lakehouse for analytical workloads, and application data such as Oracle Fusion. Some applications may also store enterprise data in non-Oracle data source or third-party catalogs. Extracting value from these diverse data estates traditionally required expert data engineers, business analysts, and administrators to work together. With the advent of AI agents, all this could be automated if only the lakehouse infrastructure can provide a single pane of glass that can (a) securely connect to and federate access across all data sources, (b) access live data without ETL, and (c) provide a semantic layer that gives agents business context.

Introducing Oracle AI Data Catalog

Today, we introduce Oracle AI Data Catalog that brings together on-premises databases, existing data lakes and lakehouses without requiring customers to replace them to become AI-ready. The data catalog functions as a central, unified data gateway to diverse data sources that provide AI agents and users with secure, governed access to live data. In addition to securely connecting Oracle data sources, the data catalog also brings Oracle enterprise security and governance to non-Oracle sources and automatically collects rich semantic information that helps AI agents efficiently derive value from all the connected data assets.

Figure 1: Introducing Oracle AI Data Catalog

The catalog is built into Oracle Autonomous AI Lakehouse and includes an Iceberg REST catalog. You can use the Oracle database engine to federate queries across data sources, such as Iceberg catalogs and other databases.

With the data catalog in Autonomous AI Lakehouse, users and agents can:

  1. Discover: Explore registered catalogs in Oracle Data Studio or through custom clients using REST APIs. It includes a fully managed Iceberg catalog, and users can connect additional catalogs and browse their schemas or namespaces and tables.
  2. Query: Use agents to query data already registered in the catalog or execute SQL queries through Autonomous AI Lakehouse using a uniform three-part name to refer to catalog tables or views: <schema>.<object>@<catalog>. We also provide a Spark connector that supports the same syntax for Spark-based query engines.
  3. Govern: Manage users, groups, and roles centrally in the catalog. Use role-based access control (RBAC) to assign privileges to catalog objects. These privileges govern requests made through catalog-connected clients and query engines; permissions on the underlying source still apply.
  4. Intelligence: Add AI-driven metadata enrichments, annotations, and analytic views to give cataloged assets business and semantic context. This context can improve AI-driven analytics. AI-driven semantic pruning helps efficiently narrow searches across hundreds of registered data sources.
  5. Trace: OpenLineage-enabled systems can report data modifications to Oracle AI Data Catalog. The catalog assembles lineage reports from across the data estate; users can query this lineage through Oracle Data Studio or REST APIs. With OpenLineage plugins configured, or with lineage collected automatically from Oracle Data Transforms and Oracle databases, users can trace provenance across lakehouse workflows.

Connecting all your data

To truly support connections across a lakehouse data estate, it is important to be interoperable with all existing catalogs. Hence, a lakehouse data catalog needs to connect to sources such as:

  1. Oracle databases with operational data across clouds or on premises
  2. Other non-Oracle databases like PostgreSQL and MySQL
  3. External structured data in object storage registered in Iceberg catalogs
  4. Other collections of catalogs, such as Microsoft Purview, AWS Glue, or Unity Catalog

This requires connections not only to database catalogs, which contain schemas and tables, but also to collections of catalogs such as Unity Catalog. We call such a collection a metalake. Data stewards can group catalogs into a metalake and connect to external metalakes as well.

Oracle AI Data Catalog provides a unified metalake: a catalog of catalogs that connects databases and other catalogs in one place. It also provides a default native Iceberg catalog to manage all your Iceberg tables in OCI Object Storage, Azure, or on-premises storage.

Discover and browse the catalog in Oracle Data Studio

The data catalog is the centerpiece of Data Studio, which can be accessed via https://datastudio.oracle.com. Administrators can provision a data catalog, connect data sources, and set up governance rules (more on this later). Authorized users can then log in to Data Studio, browse the lakehouse catalogs they can access, and run queries directly using either a built-in AI assistant or SQL Worksheet.

Figure 2: Catalog navigator in Oracle Data Studio

Govern access once and apply everywhere

Users can already apply all their governance rules at the data source when data is stored in Autonomous AI Lakehouse. However, not all data sources provide such strong governance policies. Oracle AI Data Catalog provides unified governance across all data sources, including non-Oracle sources. Administrators can bundle metadata and data access privileges into roles and assign those roles to users or groups. For ease of governance, roles can be defined centrally at different levels of the catalog hierarchy (e.g., for an entire data source or for groups of objects or tables within it) and granted to or revoked from users or groups as needed.

Access data connected to the catalog using Oracle AI Lakehouse

To access data sources listed in the catalog, there are many options available.

  • Create, update and query with Oracle AI Lakehouse using natural language or SQL with three-part catalog object identifiers. For example: SELECT * FROM SALES.ORDERS@SALES_DB
  • Use the Oracle Spark connector with your existing Spark infrastructure to query data in the catalog
  • Use an MCP tool for your AI agents to talk directly to the catalog
  • Use a custom client to directly query catalog metadata using REST APIs

All catalog access is governed centrally by policies defined by an administrator in Oracle AI Data Catalog, as described earlier. Authorized users and agents can access live data as it becomes available without any ETL.

Enrich data in the catalog with semantic and business meaning

To make AI agents effective and efficient, users can add business semantics to objects in the catalog and built-in agents automatically use that context to find relevant data in the catalog.

Catalog data can be enriched with semantic meaning in three ways:

  1. Add free-form annotations at every level of the data hierarchy: Metalake, Catalog, Schemas, Tables/Views, Columns.
    Agents can use these annotations to find relevant data.
  2. Use the AI-based enrichment feature in Oracle AI Lakehouse to automatically generate descriptions for catalogs, schemas, and objects (tables, views, and columns) in the data catalog.
  3. Create analytic views (commonly referred to as semantic views) in Oracle AI Lakehouse on top of your data, defining attribute dimensions, business metrics, and hierarchies. Users can also readily import and export the semantic views into Oracle AI Lakehouse in Apache Ossie’s portable YAML format.

Trace and understand provenance of your data with lineage

After identifying the relevant objects to query, users can check their lineage before using the data in a report to ensure it comes from a trusted source. This includes changes made as part of Oracle Data Transforms data flows, or data modifications in any monitored Oracle database. The data catalog can also be configured to consume lineage events from any third-party systems (e.g., Spark, Trino, dbt, Apache Airflow) that generate OpenLineage events. This allows data engineers to ensure that all data modifications and movements are traced in the lakehouse.

Figure 3: Sample Data Lineage for a table collected using OpenLineage

Putting it all together: An example use case

Consider an e-commerce use case in which a retailer keeps order lines in an on-premises Oracle database, customer details in a managed cloud database, and product details in Iceberg tables registered through Unity Catalog. An administrator can connect these sources to Oracle AI Data Catalog, define a data_analyst role with appropriate catalog privileges, and grant the role to analysts. An analyst can then log in through Data Studio, connect to the catalog, and run a SQL query that joins all three sources directly in Oracle Data Studio:

SELECT o.order_id, c.customer_name, p.product_name
FROM "sales"."order_lines"@ORACLE_ORDERS o
JOIN "crm"."customers"@ORACLE_CRM c
  ON o.customer_id = c.customer_id
JOIN "products"."items"@UNITY_ICEBERG p
  ON o.product_id = p.product_id;

With semantic context added to the catalog, the analyst can also ask Data Studio’s built-in AI assistant: “Show each order ID with its customer name and product name using order lines, customers, and products tables in the catalog.” The assistant can generate the three-source SQL query for review and execution. In conclusion, Oracle AI Data Catalog brings discovery, querying, access control, semantic context, and lineage into a shared catalog experience across heterogeneous sources. For an AI agent working over a multicloud lakehouse, that means a clearer path from finding relevant data to understanding and querying it through a client of your choice. Start by connecting a source and exploring the data catalog in Oracle Data Studio.

Read about Autonomous AI Lakehouse and how it can unlock your enterprise data with AI Agents.