Real-Time RAG: Live SQL, Incremental Indexing, and Freshness Tests

Short answer: Real-time RAG should not re-embed every changing record. Query current structured data live with SQL or NL2SQL, and use retrieval for documents that benefit from semantic search. For changing documents, update affected chunks incrementally, track versions and timestamps, and test whether answers prefer current evidence over stale evidence.

The common real-time RAG question is simple: “My data changes every few minutes. Should I embed it, cache it, query it, or build an agent?”

The answer depends on the data shape. Current rows are not documents. Operational data should usually stay operational. Documents, transcripts, support notes, and policies can be indexed, but they need freshness metadata and incremental updates.

Key takeaways:

• Query live structured data instead of re-embedding it on every update.
• Use incremental indexing for changing documents.
• Freshness must be evaluated, not assumed.

When you move RAG to production, when should live SQL beat retrieval?

Short answer: Use live SQL when the answer depends on current rows, counts, filters, dates, statuses, permissions, or joins. RAG is the wrong first route for questions that a database can answer directly and deterministically.

If a risk register changes every five minutes, embedding every row creates a stale copy. If the user asks for open risks, overdue mitigations, current owners, or counts, the system should query live tables.

Use this routing table:

Key takeaways:

• Do not force structured facts into a vector-only path.
• SQL has built-in advantages for current state, permissions, and joins.
• NL2SQL still needs guardrails, result limits, and inspection.

How should documents stay fresh?

Short answer: Track source timestamps, chunk hashes, embedding model versions, current-version flags, and delete state. Re-embed only changed chunks. Test stale and current versions in the same evaluation set so freshness regressions are visible.

Incremental indexing is not just a performance optimization. It is a correctness requirement. A stale chunk can produce a confident wrong answer.

A production document pipeline should record:

• source ID and source system
• source modified timestamp
• ingestion timestamp
• chunk hash
• embedding model
• current version flag
• deletion or superseded state
• tenant and ACL metadata

Key takeaways:

• Freshness metadata belongs in retrieval filters.
• Re-embedding unchanged chunks wastes compute and can create noise.
• Deletion and supersession must propagate to retrieval.

What role do caching, MCP, and tool calling play?

Short answer: Caching reduces repeated work, MCP exposes tools, and tool calling chooses actions. None of them makes stale data fresh. Each still needs permissions, timeouts, result limits, audit, and a freshness strategy.

Use caching for stable, permission-safe results. Invalidate it when source data changes. Use MCP or tool calling when the assistant needs to query a database, call an API, or run a retrieval tool. Keep each tool narrow and auditable.

Key takeaways:

• Tool calling is routing, not governance.
• MCP needs scoped tools and traceable outputs.
• Cache invalidation should be tied to source change events.

How do I implement this with Oracle AI Database?

Short answer: Use Select AI or application-controlled SQL for live structured data, Oracle AI Vector Search for document retrieval, and hybrid search where keyword and semantic search both matter. Keep source timestamps, version flags, tenant filters, and ACLs in the database path.

Useful docs and runnable assets:

• About Select AI
• Oracle AI Vector Search User’s Guide
• Understand Hybrid Search
• DBMS_VECTOR_CHAIN
• RAG evaluation notebook: oracle_rag_with_evals.ipynb

Key takeaways:

• Use live SQL for current structured facts.
• Use vector and hybrid retrieval for document evidence.
• Keep freshness filters near the data, not in the prompt.

How should I test freshness?

Short answer: Create challenge questions where old and new evidence both exist. The system should choose the current record or current document version, cite it, and ignore stale evidence unless the user explicitly asks for history.

Freshness tests should include:

• recently updated row
• stale document version
• deleted document
• superseded policy
• cached answer invalidation
• user asking for current state
• user asking for historical state

Key takeaways:

• Freshness is a user-visible quality metric.
• The evaluation set should include stale evidence on purpose.
• A system that cannot distinguish current from old evidence is not real-time.

What should I do next?

Map each question type to live SQL, document retrieval, hybrid retrieval, or agent tool use. Then build a freshness challenge set before scaling ingestion. If the system cannot pass stale-versus-current tests, do not publish it as real-time RAG.