ATLAS: What “thorough” means for each kind of node

Blog 9 · Checklist

Use this when you review a sync plan or decide which custom tests to add. Each card says what to cover and how ATLAS checks it.

Start with business cases, not node names

• A normal successful request.
• An empty or “no matches found” result.
• A missing required input.
• A validation or service-error path.
• An out-of-scope request, if the agent has scope limits.
• Each important create, update, or delete path, using safe test records.

The Opportunity Manager sessions in Oracle’s documentation used this starter set to cover listing, filtering, creating, updating, revenue actions, out-of-scope messages, and conditional routes: 15 scenarios across two workflows.

• IF and SWITCH: One test per meaningful outcome, including the false side of every IF. Checked by path assertions: nodes that must run and alternate branches that must not.
• Code and set-fields: Stable intermediate results the rest of the agent depends on. Checked by node assertions on the output, for example “NORMALIZE_TEAM_HEALTH returns the workers, summary, and status keys”.
• LLM, Agent, Multi-Agent: Output quality, grounding in the data, completeness, and no unsupported claims. Checked by a judge rubric, never by freezing generated wording. Pin structured decisions with node assertions.
• Business Objects and REST: Representative data, plus empty, missing, and error responses. Real data is recorded once and replayed; hard-to-reach responses use generated data.
• Create, update, delete, email: Every write path, with safe nonproduction records. Guardrail tests for anything with consequences. You confirm before tests change data. File mode replays the result and never repeats the action.
• Loops: The complete loop result and one representative pass through the loop body. Boundary replay plus a single-iteration test. Empty-collection tests only if the agent handles that case.
• Parallel branches: Each branch’s contribution to the joined result. Loops in separate branches share one boundary contract when the branches converge cleanly.
• Sub-workflows: The parent’s handling of the child’s result, and the child itself. The parent replays the child’s recorded result. The child gets its own tests.
• RAG and vector retrieval: Answers grounded in what was retrieved. Retrieval results replay; the judge checks grounding against them.
• HUMAN and WAIT: Approve, reject, feedback, and the timeout path. Multi-turn conversation tests that pause and resume.
• Capabilities: Each capability the workflow exposes, as its own entry point. One primary capability test plus its branches, loops, and waits.
• Repeated or bounded calls: A validation that must run once, or retries that must stop. Execution count assertions: exact, minimum, or maximum.

Create tests for the revenue update and delete paths. Use this safe test opportunity number and revenue record ID. Do not use production data. Run the focused suite and summarize the results.

Agents that pause for a person or a timer

If a workflow pauses at a HUMAN or WAIT step, ask for a multi-turn conversation test. ATLAS sends the first request, confirms the pause, and resumes with the right action.

• What to cover: For a HUMAN step: approval, rejection, and feedback when the workflow allows it. For a WAIT step: the timeout path. Separate tests for approval and rejection, so each result is easy to read. If the workflow relies on earlier context, a follow-up that uses a fact stated only in the first request.
• Good to know: HUMAN support is for the chat channel. HUMAN or WAIT inside a loop, parallel branch, or nested workflow isn’t covered yet; nor are approval processes. Retrying a live conversation repeats earlier business actions, so record in a nonproduction environment. A step that ends at a pause stays deterministic; judges apply to model-written final answers.

Create workflow conversation tests for the Request Review workflow. Cover approval and rejection separately, and add a feedback case that asks for missing business justification before approval.

The assistant runs: get-workflow-conversation-test-context · do-generate-workflow-test · run-workflow-tests

Run the test sync plan for the Scheduled Follow-Up workflow and create the missing WAIT timeout conversation coverage.