Short answer: RAG retrieves external evidence. Agent memory stores durable context about users, agents, threads, preferences, decisions, and prior work. Production agents need explicit conversation IDs, tenant scopes, provenance, deletion rules, and permission-aware recall. Without those boundaries, memory becomes prompt stuffing and state leaks across sessions.
Developers often ask memory questions while talking about RAG. That makes sense: both involve retrieval. But they solve different problems.
RAG answers, “What source evidence should the model use right now?” Memory answers, “What should this agent remember across turns, sessions, and workflows?”
Key takeaways:
- RAG is not a memory system by itself.
- A bigger context window is not durable memory.
- Production memory needs scope, lifecycle, provenance, deletion, and governance.
RAG vs agent memory: what is the difference?
Short answer: RAG retrieves documents or data to ground an answer. Agent memory persists useful state across interactions. RAG should cite source evidence. Memory should recall scoped facts, preferences, decisions, summaries, tool results, and workflow state when they remain valid and allowed.
Use this distinction:
| Capability | RAG | Agent memory |
|---|---|---|
| Primary job | Retrieve external evidence | Persist and recall useful state |
| Typical unit | Document chunk, row, source result | User preference, thread summary, decision, durable fact |
| Main risk | Wrong or inaccessible evidence | Stale, overbroad, or cross-session memory |
| Required scope | Source, tenant, ACL, version | User, agent, tenant, thread, conversation |
Key takeaways:
- RAG can be stateless.
- Memory is stateful by definition.
- Memory must be correctable and deletable.
Why does an MCP tool need a conversation ID?
Short answer: A user ID identifies the person, not the conversation. A conversation ID or correlation ID lets tools isolate state, connect logs, avoid cross-tab leakage, and prove which retrieved data belonged to which interaction.
If an LLM application calls your MCP tool without a conversation ID, your tool cannot tell whether two requests belong to the same thread or two parallel sessions. In regulated systems, that is not just inconvenient. It is a traceability problem.
Ask the calling application to pass:
- user ID
- tenant ID
- agent ID
- conversation or thread ID
- request ID
- permission context
Key takeaways:
- Conversation IDs are state isolation, not decoration.
- Correlation IDs make debugging and audit possible.
- Memory scope should come from explicit user, tenant, agent, and conversation identifiers rather than user ID alone.
What should an agent remember?
Short answer: An agent should remember durable, useful, scoped information: user preferences, task state, decisions, summaries, validated facts, and tool results that remain relevant. It should not remember secrets, transient noise, unsupported claims, or data the user is not allowed to retain.
Memory needs policy. Without policy, every interaction becomes a candidate memory, and the system becomes noisy.
Use memory categories:
| Memory type | Example | Rule |
|---|---|---|
| Preference | User prefers Python examples | Keep if useful and non-sensitive |
| Task state | Draft article awaiting benchmark data | Keep with thread scope |
| Durable fact | Project uses Oracle AI Database | Keep with provenance |
| Tool result | Retrieval run completed | Keep with timestamp and source |
| Sensitive value | Password, private key, token | Do not store |
Key takeaways:
- Memory promotion should be deliberate.
- Memories need provenance and timestamps.
- Deletion and correction are product requirements.
How do I implement this with Oracle AI Database?
Short answer: Use Oracle AI Agent Memory for persistent, scoped memory, and keep RAG, SQL, and tool outputs governed by the same access rules. Store user, agent, tenant, thread, memory, provenance, and retrieval metadata in a durable database-backed path.
Useful docs:
- Oracle AI Agent Memory getting-started guide
- Oracle AI Database Security Guide
- About Select AI
- Oracle AI Vector Search User’s Guide
For Oracle positioning, the important message is that memory is a data problem. Once memory needs persistence, scoping, retrieval, lifecycle rules, audit, and deletion, it belongs in a governed data layer rather than an ad hoc prompt or local cache.
Key takeaways:
- Use Oracle AI Agent Memory as the canonical product path for scoped persistent memory.
- Keep live facts in SQL and source-grounded evidence in RAG.
- Apply the same security rules to memory retrieval that apply to document retrieval.
How does memory affect latency?
Short answer: Memory can reduce repeated work when it recalls the right scoped context, but it can add latency if every turn performs broad retrieval. Keep memory retrieval narrow, scoped, and measurable. Use summaries for thread continuity and durable memories for facts that matter beyond the current conversation.
A practical memory stack separates:
- current prompt context
- thread summary
- durable user or task memories
- retrieved source evidence
- live tool or SQL results
Key takeaways:
- Memory retrieval should be selective.
- Thread summaries and long-term memories serve different purposes.
- Measure whether memory improves answer quality enough to justify the cost.
What should I do next?
Start by defining memory scope. Decide what is stored by user, tenant, agent, thread, and conversation. Then define promotion, update, delete, and retrieval rules. Only after that should you wire memory into an agent loop.