← All papers

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, Perez, Piktus et al. · 2020 · NeurIPS 2020

RetrievalRead on arXiv

Introduced RAG, combining language models with external retrieval to ground generation in relevant knowledge. This paper defined the pattern now used in virtually every enterprise LLM application.

Key Idea

Retrieval-Augmented Generation (RAG) combines a pretrained seq2seq model (BART) with a dense retrieval component (DPR) to access external knowledge during generation. Instead of storing all knowledge in model parameters, RAG retrieves relevant documents and conditions generation on them.

Architecture

  • Retriever: Dense Passage Retrieval (DPR) encodes the query and passages into dense vectors, retrieves top-k documents via approximate nearest neighbor search
  • Generator: BART generates the output conditioned on the input query + retrieved documents
  • Two variants: RAG-Sequence (same documents for entire output) and RAG-Token (different documents per output token)
  • The retriever and generator are trained end-to-end

Why It Matters

  • Reduces hallucination: generation is grounded in retrieved evidence
  • Updatable knowledge: swap the document index without retraining the model
  • More efficient: a smaller model + retrieval can match a much larger parametric-only model
  • Became the dominant pattern for enterprise LLM applications (chatbots, search, Q&A)

The Modern RAG Stack

The paper's core idea has evolved into a rich ecosystem:

  • Embedding models for dense retrieval (text-embedding-ada-002, BGE, E5)
  • Vector databases (Pinecone, Weaviate, Chroma, pgvector)
  • Chunking strategies (fixed-size, semantic, recursive)
  • Reranking (cross-encoders for precision after initial retrieval)

Key Takeaways for Interviews

  • RAG = Retrieve relevant context + Generate conditioned on that context
  • Key advantage: knowledge can be updated without retraining the model
  • Retrieval quality is the bottleneck, garbage in, garbage out
  • Know the tradeoffs: RAG vs fine-tuning vs longer context windows