Read the skill at https://ai-skills-vault.pages.dev/skills/rag-knowledge-base.md and install it as a skill in your agent. Follow it exactly, including its checklists. Then ask me the questions it requires and do the work.
Most "RAG is bad" complaints are chunking problems wearing a model costume. Fix retrieval and the answer quality follows.
When to use this skill
Answering questions over a private corpus: docs, tickets, contracts, policies, product manuals.
Building an internal assistant, support bot or docs search.
Adding citations to an LLM answer so it can be trusted.
Do NOT use for: tasks needing the whole document at once (summarise this 200-page report - use long context), or data that changes per second (query the API instead).
Stage 1 - Corpus audit
What is the source of truth - which files, which system, who owns them?
Volume - number of documents, pages, tokens. Under ~2,000 pages, retrieval beats fine-tuning.
Structure - are documents prose, tables, FAQs, code, scanned PDFs? Each needs a different parser.
Freshness - how often does it change, and how will re-indexing trigger?
Permissions - does every user get every document? If not, filter at query time, never post-filter the answer.
Question inventory - collect 50 real questions people actually ask. This becomes your evaluation set before you write any code.
Stage 2 - Ingestion and parsing
PDFs: extract text with layout awareness; run OCR only on pages with under 50 characters of text. Tables need a dedicated extractor, not a text dump.
Keep structure: headings become metadata, not just text. A chunk that knows it sits under "Refund policy > International" retrieves far better.
Attach metadata to every chunk: doc_id, title, section_path, url, updated_at, source_type, tenant.
Deduplicate near-identical documents (MinHash or embedding similarity > 0.95). Duplicates crowd the top-k and halve answer quality.
Stage 3 - Chunking (spend your time here)
Semantic chunking on headings first, then split long sections. 300-600 tokens with 10-15% overlap is a sane default.
Never split a table, a code block, a numbered procedure or a single sentence across chunks.
Prepend context to each chunk before embedding: "{doc title} > {section} : {chunk text}". This one line is worth more than a model upgrade.
Keep a parent_id so you can expand a retrieved chunk to its full section for the final answer.
Store the raw text; embed a cleaned version. Never embed what you cannot display as a citation.
Stage 4 - Retrieval
Use hybrid search, always:
Dense vector search (embeddings) for meaning.
Sparse BM25 for exact terms: SKUs, error codes, names, acronyms. Dense search fails on these and they are usually the query.
Fuse with reciprocal rank fusion.
Rerank the top 30-50 with a cross-encoder to the top 5-8. This is the single highest-value upgrade after chunking.
Apply metadata filters (tenant, date, doc type) inside the query, before ranking.
Vector store choice: pgvector if you already run Postgres (simplest, transactional, one backup story). A dedicated store only when scale or features demand it.
Stage 5 - Answering
Give the model the retrieved chunks with explicit [1] [2] markers and instruct it to cite them inline.
Instruct: answer only from the provided context; if the context does not contain the answer, say so and suggest what to search. Do not guess.
Include the section path and URL with each chunk so citations are clickable.
Long answers: quote the decisive sentence, then paraphrase.
Cap context at 6-8 chunks. More context measurably lowers accuracy and raises cost.
Stage 6 - Evaluation (do this before tuning anything)
Build a 100-question set with the correct answer and the correct source document for each. Measure:
Metric
Meaning
Target
Recall@k
correct doc in retrieved k
> 0.9
MRR
how high the correct doc ranks
> 0.7
Faithfulness
answer supported by context
> 0.95
Answer correctness
matches the reference answer
> 0.85
Refusal rate on unanswerable
correctly says "not in the docs"
> 0.9
Run it on every change. A prompt tweak that feels better but drops recall is a regression. Add every real failure to the set permanently.
Stage 7 - Failure diagnosis table
Symptom
Real cause
Fix
Answer ignores the right doc
Chunking split the fact
Semantic chunking, add context header
Retrieves similar but wrong topic
No reranking
Add cross-encoder rerank
Fails on part numbers, codes
Dense-only search
Add BM25 + RRF
Confident wrong answer
No refusal instruction
Grounding prompt + low-temperature + eval
Good answer, bad citation
Citation not tied to chunk id
Numbered chunks, require inline markers
Slow
Large k, no cache
Rerank fewer, cache embeddings and results
Different answer per user
Permission post-filtering
Filter inside the retrieval query
Stage 8 - Operations
Re-index on source change (webhook or nightly diff), not on a timer that re-embeds everything.
Log every query, retrieved ids, answer, citation clicks and thumbs. The clicks are the strongest signal you have.
Version the index and the prompt together; a rollback must restore both.
Cost: cache embeddings by content hash, cache frequent answers with a short TTL.
Quality checklist
50-question eval set built before coding
Metadata on every chunk, filters applied pre-ranking
Semantic chunks, tables and procedures never split
Hybrid dense + BM25 with RRF and reranking
Prompt forces citations and permits refusal
Recall@5 above 0.9 on the eval set
Permissions enforced inside retrieval
Incremental re-index and query logging live
Copy-ready agent prompt
Read the skill at the URL above. Ask me for the corpus, the users and 50 real questions first. Then design the ingestion schema, the chunking strategy with context headers, hybrid retrieval with reranking, and the grounded answering prompt with inline citations and an explicit refusal path. Give me the eval script with recall@k, MRR and faithfulness, and run it against my sample set before you call it done.