Parameters, tokens and context
Three terms that help you understand what a model stores and what it sees during a conversation.
What is it?
Parameters are the numbers a model adjusts during training; they hold what it has learned. Tokens are the pieces of text the model computes with. The context window is how much text the model can see at once during a conversation.
An everyday example
A long document can fit in the context window, and the model can still overlook an important paragraph.
Why does it matter?
These ideas explain why a model forgets the start of a long conversation, why price lists count tokens and why more parameters do not guarantee a better answer.
A common misconception
That a model knows everything it has ever read, or remembers everything you tell it. Training data, parameters and the current conversation are three different things.
In more detail
What are parameters?
Numerical values the model adjusted during training. They are not a collection of all the original texts.
What is a token?
A piece of text the model handles as one unit. It can be a whole word, part of a word or a single character.
What is a context window?
The amount of input a model can take into account at one time. A large window does not mean flawless memory.
Is memory the same as context?
No. An app can store earlier information or summaries separately; the model only takes them into account if they are given to it.
Remember: The data a model was trained on is not the same as the data it is reading now.
Reliable sources
Last reviewed 1 November 2026. An explainer, not professional, legal or medical advice. For rules and figures that change, check the official version.