Smaller models and distillation
Progress in AI is not always about making models bigger. A lot of research looks for more efficient solutions.
What is it?
Progress is not only about ever larger models. Through distillation, a large model “teaches” a smaller one that is faster and cheaper yet almost as good at certain tasks. Smaller models can even run on a phone or laptop.
An everyday example
A translator that works on your phone without an internet connection usually uses a smaller, specially tuned model.
Why does it matter?
Smaller models cut costs and energy use and allow more privacy, because the data do not leave the device.
A common misconception
That a smaller model is always worse. For a narrow task it can be just as good; for broad knowledge it usually lags behind.
In more detail
What is distillation?
A process in which a smaller model learns to imitate the useful behaviour of a larger system.
What is quantisation?
Storing a model’s numbers with fewer bits, which can reduce how much memory it needs.
Why does this matter?
Smaller models can run on smaller devices or respond faster.
What is the trade-off?
On some tasks we may lose quality, accuracy or abilities.
Remember: Efficiency is a balance between quality, cost and speed.
Reliable sources
Last reviewed 1 November 2026. An explainer, not professional, legal or medical advice. For rules and figures that change, check the official version.