RAG is a search system with a generator on the end
Retrieval-augmented generation combines a retriever that finds relevant passages with a generator that answers using them. The quality ceiling is set by the retriever: a passage that is not retrieved cannot be used, however good the model is.
Why it is like this
It separates the thing that changes often — your documents — from the thing that does not — the model. Answers become updatable and citable without retraining anything.
Not to be confused with
A way of giving the model your documents. It is an information-retrieval system, and it fails the way those fail: the right passage ranked tenth, the query phrased unlike the document, the document never indexed.
In practice
When a RAG system answers badly, instrument the retrieval step first. Most teams spend weeks on prompts for a problem that a look at the retrieved passages would have diagnosed in an hour.
Source: Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv:2005.11401) — §2. Read on .
The OWASP Top 10 for LLM Applications is © the OWASP Foundation and published under a Creative Commons licence; NIST AI 600-1 is a US Government work in the public domain. These cards are independent commentary written in our own words. ytyk.org is not affiliated with or endorsed by the OWASP Foundation or NIST.
Predict Check yourself
A RAG assistant keeps answering a question wrongly. What do you look at first?
Show the answer
The passages the retriever returned for that question — if the answer was not in them, no amount of prompt work will fix it.
The generator can only work with what it is handed. Checking the retrieved context separates a retrieval bug from a generation bug in one step.