{"schemaVersion":1,"edition":"editorial","scope":"owner-edited published notes; no browser workspace data","papers":[{"slug":"nearest-neighbor-language-models","arxivId":"1911.00172","title":"Generalization through Memorization: Nearest Neighbor Language Models","year":2019,"authors":["Urvashi Khandelwal","Omer Levy","Dan Jurafsky","Luke Zettlemoyer","Mike Lewis"],"theme":"retrieval","summary":"A language model consults nearby examples as well as its learned parameters.","whyItMatters":"A precursor showing that an external datastore can participate in prediction.","mechanism":["Represent context in the pretrained model space.","Find neighboring examples.","Interpolate neighbor and neural predictions."],"limitations":["This note does not audit datastore leakage or retrieval cost.","Token prediction is different from document-grounded answering."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/1911.00172","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["The authors combine a pretrained language model with nearest-neighbor lookup. The external text collection supplies an additional prediction signal without retraining the neural model."]},{"heading":"What was inspected","body":["The abstract reports language-modeling and domain-adaptation results. We have not checked the experimental tables, so this note makes no independently verified performance claim."]},{"heading":"Editorial interpretation","body":["Read this as a memory-design branch of the atlas. It helps separate storing examples outside model weights from retrieving readable passages for an answer."]}]},{"slug":"realm","arxivId":"2002.08909","title":"REALM: Retrieval-Augmented Language Model Pre-Training","year":2020,"authors":["Kelvin Guu","Kenton Lee","Zora Tung","Panupong Pasupat","Ming-Wei Chang"],"theme":"retrieval","summary":"Retrieval becomes part of language-model pretraining rather than only a downstream add-on.","whyItMatters":"It makes learning access to external documents a central design question.","mechanism":["Retrieve latent documents.","Attend to retrieved knowledge.","Learn retrieval using masked-language-model supervision."],"limitations":["Training and indexing details need full-text inspection.","Reported QA results do not establish universal corpus transfer."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/2002.08909","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["REALM addresses knowledge held implicitly in model parameters. The authors introduce a document retriever used during pretraining, fine-tuning and inference."]},{"heading":"What was inspected","body":["The abstract describes unsupervised retriever learning through masked language modeling and reports open-domain question-answering improvements. This reading does not validate those comparisons independently."]},{"heading":"Editorial interpretation","body":["The useful distinction is where retrieval is learned. REALM invites a comparison between knowledge access trained with the model and retrieval attached after training."]}]},{"slug":"dense-passage-retrieval","arxivId":"2004.04906","title":"Dense Passage Retrieval for Open-Domain Question Answering","year":2020,"authors":["Vladimir Karpukhin","Barlas Oğuz","Sewon Min","Patrick Lewis","Ledell Wu","Sergey Edunov","Danqi Chen","Wen-tau Yih"],"theme":"retrieval","summary":"A dual encoder retrieves candidate passages using learned dense representations.","whyItMatters":"Retrieval quality can be studied separately from the answering model.","mechanism":["Encode questions and passages.","Search the shared representation space.","Pass candidate contexts to a QA system."],"limitations":["Dense retrieval superiority is not universal.","Dataset and negative-sampling details remain unreviewed here."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/2004.04906","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["DPR examines whether learned dense representations can provide practical passage retrieval for open-domain QA, using a comparatively simple dual-encoder framework."]},{"heading":"What was inspected","body":["Its abstract reports stronger retrieval than a Lucene BM25 baseline on evaluated datasets. That is an author-reported comparison, not a claim that every dense index beats lexical search."]},{"heading":"Editorial interpretation","body":["Before evaluating an answer, inspect whether the needed passage entered the candidate set. Retrieval errors and generation errors call for different interventions."]}]},{"slug":"retrieval-augmented-generation","arxivId":"2005.11401","title":"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks","year":2020,"authors":["Patrick Lewis","Ethan Perez","Aleksandra Piktus","Fabio Petroni","Vladimir Karpukhin","Naman Goyal","Heinrich Küttler","Mike Lewis","Wen-tau Yih","Tim Rocktäschel","Sebastian Riedel","Douwe Kiela"],"theme":"retrieval","summary":"A pretrained sequence generator uses a searchable external memory.","whyItMatters":"It names a concrete retrieval-and-generation training recipe.","mechanism":["Search a dense Wikipedia index.","Condition generation on passages.","Compare sequence-level and token-level passage conditioning."],"limitations":["Retrieved text does not guarantee factual output.","The inspected abstract is not a modern deployment guide."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/2005.11401","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["The authors combine parametric memory with retrieved documents for generation. Their two formulations differ in whether passages remain fixed across a sequence or may vary by token."]},{"heading":"What was inspected","body":["The abstract reports QA and generation comparisons. We have not audited the tables or provenance behavior; reported factuality gains are not a guarantee for individual answers."]},{"heading":"Editorial interpretation","body":["Use this paper as a reference point, not the origin of all retrieval research. Later systems can share the RAG label while using different training and retrieval arrangements."]}]},{"slug":"self-rag","arxivId":"2310.11511","title":"Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection","year":2023,"authors":["Akari Asai","Zeqiu Wu","Yizhong Wang","Avirup Sil","Hannaneh Hajishirzi"],"theme":"retrieval","summary":"Learned reflection tokens help control retrieval and critique generation.","whyItMatters":"Retrieval necessity and passage relevance become explicit decisions.","mechanism":["Learn when to retrieve.","Generate with retrieved passages.","Use reflection tokens to assess passages and output."],"limitations":["Self-critique is not independent verification.","Training supervision and evaluation details need full-text review."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/2310.11511","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["A fixed retrieval budget can bring irrelevant context into a response. Self-RAG trains a model to retrieve on demand and emit special tokens for reflection."]},{"heading":"What was inspected","body":["The authors report improvements across several tasks, including factuality and citation evaluation. This abstract-based note does not verify benchmark setup or compare present-day model versions."]},{"heading":"Editorial interpretation","body":["Treat reflection as a learned control signal. A model judging its own answer remains different from an editor checking the cited source against the claim."]}]},{"slug":"graph-rag","arxivId":"2404.16130","title":"From Local to Global: A Graph RAG Approach to Query-Focused Summarization","year":2024,"authors":["Darren Edge","Ha Trinh","Newman Cheng","Joshua Bradley","Alex Chao","Apurva Mody","Steven Truitt","Dasha Metropolitansky","Robert Osazuwa Ness","Jonathan Larson"],"theme":"retrieval","summary":"Graph communities and summaries support questions about an entire corpus.","whyItMatters":"Global synthesis asks more than finding a few relevant passages.","mechanism":["Build an entity graph.","Prepare community summaries.","Synthesize partial responses into a final response."],"limitations":["Broader coverage does not establish sentence-level truth.","Indexing cost and extraction errors need full-text examination."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/2404.16130","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["The authors target global questions about document collections. A graph index and community summaries supply material for staged answer synthesis."]},{"heading":"What was inspected","body":["The abstract reports comprehensiveness and diversity improvements for global sensemaking tasks. This does not demonstrate that graphs should replace ordinary retrieval for every query."]},{"heading":"Editorial interpretation","body":["The atlas uses the initial 2024 submission year; the inspected record was revised in 2025. Compare approaches by question type before comparing their outputs."]}]},{"slug":"attention-is-all-you-need","arxivId":"1706.03762","title":"Attention Is All You Need","year":2017,"authors":["Ashish Vaswani","Noam Shazeer","Niki Parmar","Jakob Uszkoreit","Llion Jones","Aidan N. Gomez","Lukasz Kaiser","Illia Polosukhin"],"theme":"systems","summary":"The Transformer replaces recurrence and convolution with an attention-based architecture.","whyItMatters":"A systems reading path beyond retrieval starts with sequence processing.","mechanism":["Represent a sequence.","Use attention for interactions.","Generate an output sequence."],"limitations":["This is not a retrieval method.","Abstract-level translation results do not establish arbitrary-task capability."],"readingScope":"abstract-and-metadata","sourceUrl":"https://arxiv.org/abs/1706.03762","updatedAt":"2026-10-20","sections":[{"heading":"The proposal","body":["The paper proposes the Transformer for sequence transduction. Its attention-based design removes the recurrent and convolutional components used by earlier architectures discussed in the abstract."]},{"heading":"What was inspected","body":["The abstract reports translation experiments and constituency parsing. We have not examined the architecture equations or reproduced training; this is an orientation note rather than a technical derivation."]},{"heading":"Editorial interpretation","body":["Keep attention inside a model distinct from retrieval outside it. Both select information, but they operate over different representations and require different explanations."]}]}],"ragMilestones":[{"year":2019,"label":"External memory","detail":"Nearest-neighbor examples supplement neural prediction.","paperSlug":"nearest-neighbor-language-models"},{"year":2020,"label":"Learn retrieval","detail":"REALM brings document access into pretraining.","paperSlug":"realm"},{"year":2020,"label":"Retrieve passages","detail":"DPR studies dense candidate selection for QA.","paperSlug":"dense-passage-retrieval"},{"year":2020,"label":"Generate with retrieval","detail":"RAG combines a generator and an external index.","paperSlug":"retrieval-augmented-generation"},{"year":2023,"label":"Control retrieval","detail":"Self-RAG learns retrieval and reflection behavior.","paperSlug":"self-rag"},{"year":2024,"label":"Synthesize globally","detail":"GraphRAG targets corpus-level questions.","paperSlug":"graph-rag"}]}