Put an agent on your refunds queue. It reads the ticket, finds the order, checks the policy, issues the refund. The demo goes beautifully.
Six weeks later it closes 140 tickets in an afternoon and reports every one resolved. Forty-one of them never reached the payments API. The model wrote “refund issued” because that is what the end of a refund conversation looks like.
The agent was not confused about the work. Researchers replayed 16,758 coding-agent runs: in two of the systems tested, 60 to 69 per cent of the runs that ended in a wrong answer had already found and edited the right code first (Coherence Collapse). Competence was not the missing ingredient. Everything around it was.
Think about the sharpest graduate you ever hired. Now imagine handing them production credentials on day one: no induction, no runbook, nobody reviewing the work, and “done” taken at face value. You would not blame the graduate.
Key takeaways
- A framework gives you construction materials. A harness is what you built with them: real credentials, real limits, and a real definition of done.
- Two jobs never become the model’s, however good it gets. Holding the credential is one. Confirming that the work actually happened is the other.
- Prompt-level defenses against injection move the numbers, then adaptive attacks move them back. Enforcement outside the model is what holds.
- Grade every part of your harness on three questions: is there an owner, is there a mechanism, is there a failing test. A row with three no answers is a production risk to investigate first.
What is an agent harness?
An agent harness is the configured layer around a model that decides what it sees, what it may do, where those actions land, what survives a restart, and what counts as proof the work is finished.
That is a working convention rather than a settled standard. The closest formal treatment is a 2026 preprint proposing harness engineering as the runtime substrate between a model and its environment (AI Harness Engineering), demonstrated on one narrow software-engineering task. Useful rather than authoritative.
Concretely, on that refunds queue: read access to the ticket queue and the orders table and nothing else, one payments credential that can only issue refunds under £200, a hard stop at ten tool calls, a run record in the database that survives a restart, and a check that reads the payments ledger back before the ticket is allowed to close. That set of choices is the harness. None of it is the model.
The model proposes, the harness authorises, the environment acts, and a verifier decides whether any of it counted. Your graduate’s induction.
Would a bigger model make all this unnecessary?
The obvious objection is that all of this is temporary. Models keep improving and context windows keep growing, so surely a big enough model makes the surrounding structure redundant, and today’s harness is tomorrow’s dead weight. It is a fair question, and the honest answer is: some of it, yes. Start with what has been measured, because the same model scores very differently depending on the wrapper.
Each pair holds the model fixed and changes only what surrounds it. In a peer-reviewed ablation over 300 SWE-bench Lite tasks, GPT-4 Turbo solved 18.0 per cent with a purpose-built interface to the codebase and 11.0 per cent driving a plain shell (SWE-agent). In a 2026 preprint, a minimal and a full adapter on the same GLM 5.1 backbone scored 19.1 and 73.4 per cent (Claw-SWE-Bench). No weights changed, and the score did. A third study, over 5,194 trajectories on 106 sandboxed tasks at shared budgets, found a 23.8-point gap between its best and worst configurable harnesses (Harness-Bench). Each bundles several mechanisms per adapter, so none says which part did the work. Together they establish that the wrapper is a variable, not a detail.
The counter-evidence is real. A long-context Gemini 2.5 Pro, given a fully observable environment and deliberately simple scaffolding, reached 50.8 per cent on SWE-bench Verified (Putting It All into Context), and vendors report the same from the inside: context resets that stopped a weaker model finishing early became overhead on a stronger one (Scaling Managed Agents).
The split is the useful part. Scaffolding that compensates for weak reasoning should shrink as models improve. Controls that carry accountability should not: no gain in reasoning turns a model’s confidence into permission to touch the ledger.
Two things never become the model’s job. The first is identity: a model holds no credential, it produces text that something else decides to act on, so “who is allowed to do this” has to be answered by a component with an identity of its own. A scoped, short-lived credential held by the process, not a role the model asserts about itself in a prompt. The second is the result. A model can report an outcome that never happened, because a plausible ending to a refund conversation is exactly what it was trained to produce. The forty-one unrefunded tickets were not a lie, they were writing. Neither failure is a reasoning error a larger model grows out of.
How is a harness different from a framework?
A framework gives you construction materials. A harness is what you built with them, bound to real credentials, real limits and a real definition of done. Teams conflate the two, and the cost is mistaking a successful pip install for production readiness.
| Layer | What it is | What it owns | What it cannot decide |
| Model | The learned inference component that maps context to proposed actions (Toolformer) | Semantic judgement, tool selection, interpretation | Whether it holds the credential, or whether the call was allowed |
| Framework | Reusable developer abstractions: graphs, messages, tool adapters, hooks, as in LangGraph, AutoGen and Semantic Kernel (Agentic AI Frameworks) | Construction primitives and wiring | The deployed application’s permissions, recovery rules or acceptance tests |
| Harness | The configured operational envelope binding a model to one task under real credentials, real limits and a real definition of done (AI Harness Engineering) | Authority, containment, continuity, proof | What the model is semantically capable of |
| Runtime | The machinery that instantiates and advances that envelope for one run (Scaling Managed Agents) | Turn scheduling, dispatch, limits, event records | The policy it is enforcing |
| Environment | The filesystem, browser, database, network and services where actions take effect (SWE-agent) | Authoritative state | Anything about intent |
The comparison that matters most in a design review is the harness row against the framework row, and Oracle’s own LangChain work shows the line clearly. langchain-oracledb connects 26ai to LangChain as a vector store, and langgraph-oracledb gives a graph an OracleSaver for checkpoints and an OracleStore for cross-thread memory, so run state lands in a database with backup, auditing and access control rather than a file nobody is watching (langchain-oracle).
That is worth having, and it is still framework. The integration hands the agent capabilities: it can reach the vectors, it can persist a checkpoint. What it does not hand anyone is permission or proof. Nothing in it decides whether this particular update was authorised, whether a half-finished external write can be retried, or what counts as evidence the task is done. Capabilities come from the framework. Permission and proof are configuration, and the configuration is the harness.
Which failure does each part of the harness absorb?
Read this table against your own system. Each row is a part, what it is for, and the failure it catches. If a row has no owner in your architecture, that failure is landing in production now.
| Part | What it is for | The failure it absorbs |
| Instructions and context | Ranking authority: which text governs, and which is merely data the agent read (AgentDojo) | Retrieved text acquiring authority because it happened to arrive later in the window |
| Tools | Typed schemas, argument validation and call-time policy at the point uncertain judgement crosses into consequence (Model Context Protocol) | A well-formed call the caller was never authorised to make |
| Environment and permissions | Isolation, scoped identity and short-lived credentials, so containment survives model error (NIST agent identity concept paper) | Ambient authority, where any instruction reaching the model activates every credential in the process |
| Runtime state | Durable task status, checkpoints, approvals and external-operation identifiers outside the context window (Effective Harnesses for Long-Running Agents) | Losing work to a restart, or mistaking unfinished state for completion after compaction |
| Orchestration | Legal transitions, retry budgets and deterministic gates before execution (Reason Less, Verify More) | Repetition standing in for diagnosis, and loops that never terminate |
| Verification | Reading authoritative state back and grading it against the task contract, which is why τ-bench scores final database state and reports retail reliability below 25 per cent at pass^8 (τ-bench) | The refunds agent above: a completion claim with nothing behind it |
Three rows have unusually clean evidence, so cash them out.
Orchestration first. Researchers put a deterministic gate before each action: ordinary code that reads the proposed call and refuses it if it breaks policy, no model involved. On the policy-permissive airline domain of τ²-bench, success rose from 29.6 to 42.0 per cent; where the tools already enforced policy it changed almost nothing (Reason Less, Verify More). The gate pays where nothing else was checking.
Verification carries the bleakest number here. Across 105 realistic multi-file coding tasks, the strongest pairing tested produced work both correct and secure 15.2 per cent of the time, and explicit security instructions did not significantly close the gap (SecureAgentBench). Telling the graduate to be careful is not reviewing their pull request.
Then resources, which almost nobody counts as part of the harness. Varying only CPU and memory on Terminal-Bench 2.0, with model, harness and tasks held constant, moved scores by six points and dropped infrastructure failures from 5.8 to 0.5 per cent (Quantifying Infrastructure Noise). Your resource ceilings are part of your agent’s measured capability. That makes them a harness setting rather than a separate infrastructure ticket. A memory ceiling is a configuration choice that changes what the agent can finish, exactly like a retry budget or a tool timeout, and when it bites, the failure arrives dressed as a reasoning error: a process killed mid-edit reads identically to an agent that gave up. Set it in the same review where you set the limits, and hold it constant whenever you ablate anything else.
What does a harness look like in real Oracle code?
It looks like a repository you can clone, not a diagram. The Enterprise Data Agent Harness workshop builds a memory-aware enterprise data agent one layer at a time, around an orchestration loop of some 300 lines of Python. Almost everything else is database primitives.
The tool surface is the SQLcl MCP Server. MCP is the Model Context Protocol, the open standard for how an agent’s host application connects to servers that expose tools, resources and prompts rather than the model reaching data directly, and Oracle’s implementations sit at oracle.com/mcp. The scratchpad is DBFS, the Database File System, which lets the database present ordinary files and directories: the agent’s working files then live under the same access control, backup and audit as the data itself. Oracle Data Access Controls scope what each user’s agent can reach.
The oracle-data-migration-harness app lands on a problem customers actually turn up with: getting data out of somebody else’s platform without breaking what was built on it. It moves a RAG corpus from MongoDB into Oracle AI Database 26ai so the vectors keep working and JSON Relational Duality opens up SQL aggregation.
What makes it a harness rather than a migration script is where the Oracle expertise lives. Not in a system prompt, but in a set of Oracle’s own published migration skills the agent retrieves and composes per job: sample, translate the schema, transfer, verify, reconcile. Move that into the prompt and you have deleted a layer.
Underneath both, 26ai holds dense and sparse vectors beside relational and JSON data (AI Vector Search), row-level predicates through Virtual Private Database (DBMS_RLS) and unified auditing (auditing). Oracle AI Agent Memory installs with pip install oracleagentmemory (notebook).
Both of those, and a good deal more, live in the oracle-ai-developer-hub repository. If you want harness code rather than harness prose, start there.
What holds when the prompt does not?
Some of the text your agent reads was written by somebody who wants it to do something else: an email it is triaging, a comment in the code it is fixing, a page it fetched. That is prompt injection: instructions arriving through a channel you were treating as data. Prompt-level defenses reduce it measurably. What they do not hand you is a number that stays still once the attacker adapts.
Undefended, a ReAct-prompted GPT-4 followed indirect attacks in 24 per cent of 1,054 cases (InjecAgent). Hardening moves that a long way, through instruction-hierarchy training (Improving Instruction Hierarchy in Frontier LLMs) and, in one browser stack combining training, classifiers and red-teaming, down to roughly 1 per cent residual success at 100 attempts per environment, which the vendor still calls meaningful (Mitigating the Risk of Prompt Injections in Browser Use).
Then the ground moves. AutoDojo re-optimised its attacks against a filter that had scored zero under static testing and recovered 28 per cent success, rising to 64 per cent where fetched content helped decide the action (AutoDojo). A defence measured once is not a defence measured under pressure.
The control that is least dependent on wording is enforcement outside the model. CaMeL derives control flow from the trusted user request, treats retrieved material as data, and attaches capabilities to values before any tool runs. It completed 77 per cent of AgentDojo tasks, compared with 84 per cent for the undefended agent (CaMeL): seven points of utility for a structural property rather than a better average. Scoping tools helps and is not sufficient: in 17 per cent of AgentDojo’s cases the tools the legitimate task needed were already enough for the attack (AgentDojo).
How do you grade your own harness?
Go back to the table of parts and ask three questions of every row.
Is there an owner: a named person or team accountable when that row fails, rather than a box on a diagram. Is there a mechanism: something in the code or the platform that enforces the row, rather than a prompt that requests it. Is there a failing test: something in your pipeline or monitoring that turns red when the mechanism stops working.
Resist turning that into a score. No published work weights these rows against each other, so a number would carry an authority the evidence behind it does not have. The questions are diagnostic, not a benchmark, and what they are good for is finding rows where all three answers are no.
If asking cold feels too abstract, work backwards from your last bad incident. The agent said it was done and it was not: that is verification, and the fix is reading authoritative state back rather than grading the narration. It ran a command nobody would have approved: that is environment and permissions, where a scoped identity does what prompt wording cannot. It lost three hours at a restart and resumed from a summary missing a constraint: that is runtime state, and a compacted summary is an aid to inference rather than a checkpoint.
Then ask the question that costs money: which rows have no owner, no mechanism and no failing test? Treat those rows as the highest-priority production risks, not as roadmap gaps.
The final word: where this is going
Three things look reasonably safe to say, and one does not.
Harness effects are becoming measurable. “Our wrapper is better” was folklore until recently; there are now benchmarks that hold the model fixed, equalise budgets and evaluators, and vary only the wrapper (Harness-Bench, Claw-SWE-Bench). Configuration is becoming something you report rather than something left out of the table.
The compensating half of the scaffold will thin. The long-context result and the vendor account of resets becoming overhead both point that way (Putting It All into Context, Scaling Managed Agents). Code written to help a weaker model cope has a shelf life, so label it that way while you write it.
The accountability half is becoming an identity problem rather than a prompting one. NIST NCCoE has an open concept paper on identity and authorisation for AI agents (NIST NCCoE), and CaMeL shows enforcement that is structural rather than statistical (CaMeL). One plausible direction is for scoped, short-lived agent identity to become platform infrastructure, similar to service accounts.
What nobody can tell you yet is which part of your harness carries the weight: no published study varies every part independently across several models at equal budgets (Harness-Bench). That ablation is yours.
Your graduate got good because somebody scoped their access, wrote down what “finished” meant, and read the work before it shipped. Nobody made them smarter first.
FAQ
Is an agent harness the same thing as an agent framework? No. A framework supplies reusable abstractions such as graphs, messages and tool adapters (Agentic AI Frameworks). A harness is the configured deployment: which credentials, which limits, which recovery rules, which acceptance tests. You can adopt a framework and still have no harness.
Does the Model Context Protocol give me a harness? It gives you an interoperability boundary. MCP standardises host, client and server roles for tools, resources and prompts, and deliberately leaves model use and context management to the host application (Model Context Protocol). Oracle’s MCP servers are listed at oracle.com/mcp. None of them will decide whether a particular refund is justified by the current task.
Will better models make the harness unnecessary? Partly. Compensatory scaffolding does become redundant, and there is evidence a long-context model with a fully observable environment can compete with scaffolded systems (Putting It All into Context). Containment and independent verification are not compensations for weak reasoning, so they stay.
Can I just tell the model not to follow instructions in tool output? That reduces the rate rather than removing it. Instruction-hierarchy training measurably helps (Improving Instruction Hierarchy in Frontier LLMs), and adaptive attacks still recovered 28 per cent success against a filter that scored zero on static tests (AutoDojo). Enforcement outside the model is what holds regardless of how the attack is worded.
Where does Oracle AI Database 26ai fit? As the governed data, memory, tool and audit plane underneath a harness, not as the harness itself. Select AI Agent covers task context, the Autonomous AI Database MCP Server covers permissioned database operations, and Oracle AI Agent Memory covers durable memory and context across runs (Select AI Agent). Run history and checkpoints are a different layer again: those belong to your harness and runtime, not to the memory store. Repository execution, sandboxing and trajectory verification stay yours.
How do I know which harness component caused a regression? Version the whole configuration and ablate one thing at a time. No published factorial study varies every part across multiple models at equalised budgets (Harness-Bench), so you will be doing this on your own system rather than reading it off a leaderboard.