# Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin

2017 | arXiv:1706.03762

Source: https://arxiv.org/abs/1706.03762

Reading scope: abstract-and-metadata

Editorial update: 2026-10-20

The Transformer replaces recurrence and convolution with an attention-based architecture.

## Why this matters

A systems reading path beyond retrieval starts with sequence processing.

## The proposal

The paper proposes the Transformer for sequence transduction. Its attention-based design removes the recurrent and convolutional components used by earlier architectures discussed in the abstract.

## What was inspected

The abstract reports translation experiments and constituency parsing. We have not examined the architecture equations or reproduced training; this is an orientation note rather than a technical derivation.

## Editorial interpretation

Keep attention inside a model distinct from retrieval outside it. Both select information, but they operate over different representations and require different explanations.

## Mechanism

Represent a sequence.

Use attention for interactions.

Generate an output sequence.

## Limits

This is not a retrieval method.

Abstract-level translation results do not establish arbitrary-task capability.