The transformer: how does a model connect words?
The transformer is a highly influential neural-network design for processing sequences.
What is it?
The transformer is a neural network design introduced in 2017 that powers most of today’s language models. Its key ingredient, the attention mechanism, works out for each word which other words in the text matter most to it.
An everyday example
In the sentence “The cat did not jump onto the table because it was too big”, the model has to work out whether “it” means the cat or the table. Attention helps it link the right words.
Why does it matter?
The transformer made it possible to train on huge amounts of text, and with that came today’s chatbots, translators and coding models.
A common misconception
That “attention” means the model pays attention as a person does. It is a mathematical operation, not awareness.
In more detail
What is attention?
A mechanism the model uses to calculate which parts of the input matter for what it is processing right now.
Why does this matter?
It lets the model flexibly take into account the connections between parts of a text and other data.
Is a transformer the same as an LLM?
No. Many large language models are built on transformers, but there are other designs too, and transformers have other uses.
Does “attention” mean human attention?
No. It is a mathematical operation, not evidence of consciousness or inner experience.
Remember: “Attention” in the mathematical sense is just a technical name.
Reliable sources
Last reviewed 1 November 2026. An explainer, not professional, legal or medical advice. For rules and figures that change, check the official version.