Short answer: RAG chunking fails when it cuts content away from the context that makes it meaningful. A chunk should preserve headings, table headers, source IDs, timestamps, speaker labels, media traits, and security metadata. Good parsing and chunking make retrieved evidence understandable before the model ever sees it.
Most RAG tutorials spend a few minutes on chunking and then move on to embeddings. In production, chunking is often where retrieval quality is won or lost.
The model cannot use evidence that was damaged before retrieval. If a table row loses its headers, a transcript loses the speaker, or a bullet loses its parent heading, the retriever may find text that looks related but cannot support an answer.
Key takeaways:
- Chunking is retrieval design, not housekeeping.
- The goal is not equal-sized text. The goal is evidence that can stand alone.
- Parsing quality matters as much as embedding quality.
Why does chunking break a RAG application with vector search?
Short answer: Chunking breaks RAG when boundaries ignore meaning. Fixed-size chunks can split sentences, detach bullets from headings, cut tables mid-row, and separate numbers from labels. The retriever then returns fragments that are semantically close but incomplete.
The common failure is a chunk that is technically relevant but useless. For example, a chunk that says “up to 30 days” is not answerable unless it also carries what the 30 days refers to.
Structure-aware chunking should preserve:
- document title
- section and parent headings
- list context
- table headers
- page or slide number
- source path
- version and timestamp
- tenant and access metadata
Key takeaways:
- Inspect retrieved chunks from failed queries before changing models.
- Attach parent context to small chunks.
- Keep metadata with the chunk, not in a separate system that retrieval cannot filter.
How should I chunk tables and spreadsheets?
Short answer: Keep table headers, row labels, sheet names, and units with every retrieved table fragment. Do not let a row become a list of disconnected numbers. If a table is central to the answer, store a text representation and structured fields that can be queried directly.
Tables are hard because their meaning is relational. The value 12 means nothing without the column name, row label, unit, and sometimes the preceding section.
Use a table strategy:
| Table problem | Safer approach |
|---|---|
| Row loses column headers | Repeat headers in each table chunk |
| Sheet context disappears | Include workbook, sheet, and section names |
| Numeric values need filtering | Store structured fields for SQL |
| Table spans pages | preserve continuation markers and page references |
Key takeaways:
- Table RAG often needs both text retrieval and structured querying.
- Repeat headers deliberately; do not rely on proximity.
- Preserve provenance so citations can point to the real source.
How should I parse PDFs?
Short answer: Treat PDFs as layout artifacts, not clean documents. Parse text, headings, reading order, tables, page numbers, and source coordinates where possible. Test parsing output before embedding it, because a bad PDF extraction can make every downstream retrieval method look worse than it is.
PDFs can have multi-column layouts, footnotes, tables, captions, scanned pages, and broken reading order. If parsing turns a policy into scrambled text, embeddings will faithfully index the mess.
A production parser should produce:
- clean text
- reading order
- table boundaries
- headings and hierarchy
- page references
- image captions or extracted descriptions when needed
- source IDs for citation
Key takeaways:
- Do not benchmark retrieval until extraction quality is visible.
- Keep page and source references for citations.
- Use a challenge set with tables, scanned pages, and multi-column layouts.
How should I chunk transcripts and media summaries?
Short answer: Keep timestamps, speaker labels, cleaned text, summary fields, and raw references together. For video or creator search, evaluate extracted traits as structured metadata before using vectors to rank results.
Raw transcripts are long, repetitive, and full of filler. Summary-only storage loses exact quotes. The practical path is both: cleaned chunks for retrieval, source timestamps for audit, and summaries or topics for navigation.
For media RAG, extracted traits should be tested. If the app needs to find “curly haired creator” or “risk discussion at minute 42,” those labels need an evaluation set. Vector similarity is not a yes/no trait detector.
Key takeaways:
- Speaker and timestamp metadata are part of the evidence.
- Store raw references even when retrieval uses cleaned text.
- Evaluate extracted media traits before ranking on them.
How do I implement this with Oracle AI Database?
Short answer: Use Oracle AI Database to keep chunks, embeddings, metadata, and retrieval filters close together. Use DBMS_VECTOR_CHAIN for text processing and chunking workflows, Oracle AI Vector Search for embeddings and similarity search, and hybrid search when lexical and semantic retrieval both matter.
Useful docs and runnable assets:
- DBMS_VECTOR_CHAIN
- Oracle AI Vector Search User’s Guide
- Understand Hybrid Search
- RAG evaluation notebook: oracle_rag_with_evals.ipynb
Key takeaways:
- Keep chunk text and metadata in the same governed retrieval path.
- Version chunking configuration so retrieval changes can be compared.
- Test tables, PDFs, transcripts, and media separately because each fails differently.
What should I do next?
Build a chunking evaluation set before tuning chunk size. Take failed user questions, inspect the retrieved chunks, and identify whether the missing evidence was a parsing problem, a boundary problem, a metadata problem, or a retrieval-ranking problem. That diagnosis is faster than changing the model and hoping the corpus improves.