Why pointing general-purpose LLMs at enterprise data is not an Enterprise AI strategy
Ask an LLM why operating margin fell in EMEA last quarter.
Point it at the warehouse, hand it the ledger, and watch it go. Twelve seconds later you have a structured, fluent, confident answer—often before Finance has finished asking who gave it access.
It answers with the confidence of a new hire on day one. Charming, articulate, and not yet aware of how anything here actually works.
The rest of this post is about that answer. Specifically, about why you cannot tell from reading it whether it is right.
What is the difference between data access and business understanding?
Data access is a technical capability. Connectors, generated queries, retrieval services, and agent tools can put enterprise records in front of a model in minutes. Business understanding is what those records mean inside your company. The metric definitions, account hierarchies, reporting policies, exceptions, and operating rules that turn raw rows into a business answer. Access moves data to the model. Understanding tells the model what the data means. They are not the same thing, and the first one is the easy part.
Models provide reasoning. Connectors provide access. Neither provides an understanding of how your business works.
Between the margin question and the confident answer, the system made a series of choices. It picked sources to treat as authoritative. It constructed joins. It selected a definition of margin. It applied a hierarchy and a reporting period. It decided what you were allowed to see. If those choices were not inherited from governed context, they were assumptions, made silently, somewhere in the path from prompt to response.
Connectivity does not create comprehension either. Fluency just makes the gap harder to see.
Why can’t the model explain your operating margin?
Go back to the EMEA answer and start pulling threads. The ledger contains numbers, but the answer does not live in the numbers alone.
To explain margin, the system needs your chart of accounts. Which segments represent legal entities, cost centers, regions, products, and natural accounts. It needs the right account hierarchy, in the right version for the reporting period. It needs to know whether “margin” means the statutory view or the management view, how intercompany activity is eliminated, whether results are shown at actual or constant currency, and which late adjustments made the cut.
None of this is a technical footnote. It is the business definition of the answer. Oracle Fusion Cloud Financials documentation runs pages deep on chart of accounts components. Segments, labels, validation rules, security rules, versioned hierarchies. Raw journal rows explain none of it to a model.
An LLM may know accounting. It does not know how your company performs accounting unless that meaning is explicitly available to it.
So the model does what models do. It completes the pattern. It produces a mathematically valid answer to some version of the question, which may or may not be your version. A plausible number and the correct number look identical in a chat window. That is the whole problem in one sentence.
Can’t you just put the context in the prompt?
You can, and people do. Use this margin definition. Exclude these entities. Follow this hierarchy. Trust this source. A knowledgeable analyst can coax the model to the right answer almost every time.
Notice who is doing the understanding.
“If the business has to be re-explained in every prompt, the model does not understand the business. The prompt writer does.”
That makes business meaning temporary, manual, and dependent on whoever is typing. A different analyst uses a different definition. A rushed user skips an exception. A tired expert forgets that the hierarchy changed last quarter. The model may not flag the missing fact. It completes the pattern and sounds exactly as confident as it did when the prompt was perfect.
It may also be expensive. In an earlier post, Why Your AI Bill Is Really a Context Bill, I argued that the answer is cheap and the conversation is expensive. Rebuilding context prompt by prompt burns time and tokens to buy an understanding that evaporates when the chat ends. Prompt libraries help. They are still not a semantic layer.
Why does business context have to span the enterprise?
Wiring the model into one application’s semantic model is better than pointing it at raw tables. But the questions executives actually ask refuse to stay inside one application. Customer profitability alone can cross CRM identity, commerce orders, supply-chain fulfillment, service costs, ERP payments, and Finance’s management definitions, each with its own idea of who the customer is.
What enterprise AI needs is business context that persists and travels. Governed definitions for entities, metrics, and KPIs. The relationships and hierarchies that explain how the business is organized. The policies, authority, lineage, and access controls that govern who can see and do what. And current state, so durable meaning is applied to what is true now, not what was true at the last export. None of that means copying every byte into one place. It means the same meaning is available wherever analytics, models, agents, and workflows touch the data.
“Durable business context is no longer optional. It is what keeps enterprise AI affordable and its answers trustworthy.” — Why Your AI Bill Is Really a Context Bill, 2026
Analysts now advise enterprises to treat a context layer as core data and analytics infrastructure and argue that assistants, copilots, and agents should share one semantic foundation so they interpret business terms consistently. The terminology varies—semantic layers, ontologies, knowledge graphs. The conclusion does not. Raw access is not enough.
What does the architecture look like when it works?
The fix is not a better model. Model quality is improving everywhere, more or less simultaneously, and whichever one you pick today isn’t necessarily what you’ll be using within a year. The differentiated part is what surrounds the model.
Oracle AI Data Platform is built on that premise. It brings trusted enterprise data, deep business semantics, and AI together in the flow of work, creating a system of context—persistent, shared business meaning applied to live state and reused across analytics, models, agents, and workflows. Systems of record stay authoritative. They are extended, not replaced. People still define outcomes, policies, and authority, with approvals and auditability built into consequential workflows.
The model becomes a choice. The understanding becomes durable.
How do you know whether you have true enterprise AI?
The next time an AI tool hands you an impressive answer, do not ask only whether it sounds right. Ask what it inherited.
Which business definitions, hierarchies, and policies shaped the answer?
Which sources did it treat as authoritative?
What assumptions did it make while joining and interpreting the data?
Can the answer be reproduced, explained, governed, and audited?
If nobody can say, the organization has AI access to data. It does not yet have true enterprise AI.
Models provide reasoning. Connectors provide access. Business context provides understanding.
Without it, you have a very articulate visitor wandering through the data estate on a temporary badge and suspiciously good formatting.
It will happily explain your margins to you. Nobody has explained your margins to it.