Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela
9 min
Large language models store substantial world knowledge in their parameters, but they struggle to expand or update this knowledge, provide provenance for their answers, and avoid hallucinations. The authors explore whether end-to-end training of models that combine parametric memory (a pre-trained seq2seq generator) with non-parametric memory (a dense vector index of Wikipedia) can improve performance on knowledge-intensive NLP tasks.
The proposed RAG model utilizes a bi-encoder architecture, specifically a pre-trained retriever (DPR) to query a dense vector index of Wikipedia and a pre-trained BART-large transformer to generate responses. The model treats retrieved documents as latent variables and marginalizes over them. The authors introduce two formulations: RAG-Sequence, where the same retrieved document is used for the entire generated sequence, and RAG-Token, which allows the generator to select different documents for each generated token. Both components are fine-tuned jointly without requiring intermediate supervision on the documents retrieved.
RAG models set new state-of-the-art results on several open-domain question answering benchmarks, outperforming both purely parametric models and specialized retrieve-and-extract architectures. Unlike previous extractive systems, RAG demonstrates superior performance by generating answers even when the correct information is not verbatim in any retrieved passage. In abstractive generation and question generation tasks, human evaluations indicate that RAG outputs are more factual and specific than those of a standard BART baseline. Furthermore, the model's non-parametric memory can be 'hot-swapped' at test time to reflect updated world information without retraining the generator.
This work provides a robust framework for hybrid models that overcome the limitations of fixed, parametric knowledge storage. By enabling models to access and cite external, updatable evidence, RAG offers a scalable path toward more interpretable and reliable language systems. It demonstrates that combining pre-trained retrieval with generative architectures is highly effective for tasks where human-level performance depends on external knowledge sources.
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures. Additionally, providing provenance for their decisions and updating their world knowledge remain open research problems. Pre-trained models with a differentiable access mechanism to explicit non-parametric memory can overcome this issue, but have so far been only investigated for extractive downstream tasks. We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) -- models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, one which conditions on the same retrieved passages across the whole generated sequence, the other can use different passages per token. We fine-tune and evaluate our models on a wide range of knowledge-intensive NLP tasks and set the state-of-the-art on three open domain QA tasks, outperforming parametric seq2seq models and task-specific retrieve-and-extract architectures. For language generation tasks, we find that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.
Sam: The search component is the Dense Passage Retriever, which matches questions and passages using numerical representations called vectors. The generator is BART, a pre-trained text-to-text model. The external collection contains about twenty-one million passages split from Wikipedia articles.
Alex: You said it learns to consult documents. What gets trained, and who tells it which passage is correct?
Sam: During task-specific training, nobody supplies the correct passage. The model learns from input-output pairs, such as questions and answers. The query encoder, which turns the input into a search representation, and the generator are updated together.
Alex: That sounds like a substantial reduction in annotation. Is it entirely free of supervision about retrieval?
Sam: No. The retriever starts from a model trained with answer-containing passages for Natural Questions and TriviaQA. The lack of direct evidence supervision applies to RAG’s task-specific fine-tuning. Also, the document representations and search index stay fixed during that training.
Alex: There are two versions in the paper. What distinction should someone remember without following the mathematics?
Sam: RAG-Sequence scores a whole answer under each retrieved passage, then combines those scores. RAG-Token combines passage-conditioned predictions at each output step. That lets different passages contribute to different words in the same response.
Alex: Can you make that difference concrete with something the authors actually generated?
Sam: Take their question-generation example with the input “Hemingway.” The task is to write a Jeopardy-style clue whose answer is that name. The generated clue connects “The Sun Also Rises” with “A Farewell to Arms.”
Alex: Those titles could come from different passages. Do the authors show that happening, rather than just infer it from the finished clue?
Sam: They examine which passages contribute during generation. One becomes more influential around “The Sun,” another around “A Farewell to Arms.” After a title begins, dependence on a particular passage weakens.
Alex: That suggests retrieval supplies the cue, while the model’s stored knowledge helps finish the title.
Sam: That is the authors’ interpretation. They also show that BART can complete those titles when given partial text. It’s a concrete example of external knowledge guiding information already held in the generator.
Alex: Let’s separate that illustration from the main evidence. How much better were the answers?
Sam: On Natural Questions, RAG-Sequence achieved roughly forty-five percent exact-match accuracy. The comparison system, Dense Passage Retriever with an extractive answering pipeline, achieved roughly forty-two percent. Exact match means the generated answer must match a reference answer under the evaluation rules.
Alex: A useful gain, but far from reliable answering. Was the improvement confined to that benchmark?
Sam: No. The paper reports leading results on several open-domain answering benchmarks, where the system must find its own evidence. TriviaQA needs a qualification: RAG leads on the comparison split used for another generative model, not the standard split against the extractive system.
Alex: Answer matching doesn’t tell us whether longer text is factual. What evidence supports that part of the claim?
Sam: For Jeopardy-style clue generation, evaluators compared RAG-Token with BART across four hundred and fifty-two pairs. They judged RAG more factual in about forty-three percent of cases. BART was more factual in about seven percent.
Alex: That’s a preference comparison, not a claim that nearly every RAG clue was correct.
Sam: Correct. Other judgments included both good, both poor, and no majority. Evaluators also preferred RAG for specificity. My reading is that this supports improvement over BART on this task, not a general factuality guarantee.
Alex: And does the learning part matter, or could you simply attach a fixed search engine?
Sam: Freezing the retriever reduced performance across the evaluated tasks. But learned dense retrieval wasn’t universally best. A word-overlap search method performed best on fact verification, which the authors suggest may reflect that task’s emphasis on named entities.
Alex: The replaceable knowledge store seems especially practical. Did they test whether changing it actually changes the answers?
Sam: They tested eighty-two world leaders whose officeholders changed between their Wikipedia snapshots. With the newer index, the unchanged model answered about sixty-eight percent correctly for the newer officeholders. Using an index from the wrong period sharply reduced accuracy.
Alex: That demonstrates an update mechanism, but on a narrow test. It doesn’t establish that every outdated belief disappears when you replace the documents.
Sam: That’s the boundary I’d keep in mind. The update experiment concerns world leaders, and the broader experiments use Wikipedia. The paper itself notes that Wikipedia can be incomplete or biased, and some evaluation questions cannot be answered from it alone.
Alex: Who should read the full paper, and where should they spend their attention first?
Sam: Researchers building knowledge-intensive generation systems should start with Methods, especially the difference between sequence-level and token-level evidence. Then read Retrieval Ablations and Index hot-swapping. Those sections show when learned retrieval helps, where simpler search competes, and what updating knowledge actually demonstrated.
Alex: And for someone who mainly wants the idea to carry into another field?
Sam: A model doesn’t have to memorize every fact it uses. Some knowledge can live in documents you can inspect and replace—but retrieval is support for an answer, not a guarantee of truth.
Alex: Thanks for listening.