# Generalization through Memorization: Nearest Neighbor Language Models

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis

2019 | arXiv:1911.00172

Source: https://arxiv.org/abs/1911.00172

Reading scope: abstract-and-metadata

Editorial update: 2026-10-20

A language model consults nearby examples as well as its learned parameters.

## Why this matters

A precursor showing that an external datastore can participate in prediction.

## The proposal

The authors combine a pretrained language model with nearest-neighbor lookup. The external text collection supplies an additional prediction signal without retraining the neural model.

## What was inspected

The abstract reports language-modeling and domain-adaptation results. We have not checked the experimental tables, so this note makes no independently verified performance claim.

## Editorial interpretation

Read this as a memory-design branch of the atlas. It helps separate storing examples outside model weights from retrieving readable passages for an answer.

## Mechanism

Represent context in the pretrained model space.

Find neighboring examples.

Interpolate neighbor and neural predictions.

## Limits

This note does not audit datastore leakage or retrieval cost.

Token prediction is different from document-grounded answering.