How does an AI model learn from its mistakes?
A model learns by measuring its mistakes and nudging its settings. After each guess, a number called the loss says how wrong it was. Backpropagation works out which way each of the billions of weights should turn to lower the loss, and gradient descent nudges them all a small step that way, over and over.
▶ Try it
Why it matters
“Training” sounds mysterious, but it's one loop: score the mistake, find which way is downhill, nudge every dial a little. How big each nudge is decides whether training works at all.
In 6 minutes you'll be able to
- The loss is one number for how wrong a guess was. Training tries to push it down.
- Backpropagation finds the downhill direction for every dial, and gradient descent nudges them all.
- Too small a learning rate crawls, too big a one overshoots or explodes, so engineers tune it carefully.
What you'll practise
Four words engineers use
Tap each card. These four ideas are the whole training loop.
- 📉 Loss
- One number for how wrong the model was. If the real next word was “blue” and the model gave it a low chance, the loss is high.
- 🎛️ Weights
- The model's dials, billions of them. Together they decide every prediction. Training only ever changes these numbers.
- 🧭 Gradient
- For every dial: which way to turn it to lower the loss, and how much it matters. Backpropagation works this out for all dials at once.
- 👣 Learning rate
- How big each nudge is. Too small and training crawls, too big and it overshoots or blows up.
Roll down the loss valley
The ball is one of Vireo's dials. Choose a step size, press Nudge, and get the loss under the green line.
Step size (learning rate): tiny · small · medium · big · huge
The ball's height is the loss. Each Nudge checks which way is downhill and turns the dial that way, by the step size you choose. Get the loss under the green line.
- On this valley medium steps are quickest. Tiny ones crawl, big ones bounce, huge ones fly out.
- A much steeper valley. Will the same step size still work? On a steep valley, small steps win. A step that was fine before now overshoots, which is why engineers tune the learning rate.
- Now a long, gentle slope. What suits this one? On a gentle slope, big steps are safe and save time. Real training runs usually make their steps smaller as they go.
Read the training dashboard
Vireo's loss line jumps up and down wildly and then shoots off the chart. What's the likeliest fix?
- Lower the learning rate (Answer)
- Raise the learning rate
- Delete half the dials
Which part works out which way every single dial should turn?
- Backpropagation, which computes the gradient (Answer)
- An engineer who checks each dial by hand
Does a lower loss mean Vireo is now always right?
- No, it means its guesses fit the training text better (Answer)
- Yes, a low loss means it has stopped making mistakes
Remember
Training is one loop: measure the loss, find which way is downhill, nudge every dial a small step. The step size decides if it works.
Loss measures the mistake, the gradient points downhill, and the learning rate sets the size of each nudge.
Key facts
- Backpropagation, the method that works out how to adjust every weight in a network, was popularised in a 1986 paper in Nature. · Source: Rumelhart, Hinton & Williams, Learning representations by back-propagating errors, 1986
- GPT-3 has 175 billion parameters, and every one of them is a dial that training adjusts. · Source: Brown et al., Language Models are Few-Shot Learners, OpenAI, 2020
- Most large language models are trained with Adam or a close variant, a refined form of gradient descent that adapts the step size for each dial. · Source: Kingma & Ba, Adam: A Method for Stochastic Optimization, 2015
- Large training runs typically warm the learning rate up at the start and lower it gradually as training goes on.
Updated:
Questions people ask
What is gradient descent in simple terms?
A way of finding good settings by taking small steps downhill. You check which direction lowers the error, take a small step that way, and repeat until the error is low.
What is a loss function?
A formula that turns a model's mistake into one number. For a language model it measures how surprised the model was by the real next token: the less likely it thought that token was, the higher the loss.
What is a learning rate?
The size of each update step during training. Too high and training becomes unstable, too low and it takes far too long, so it is one of the most carefully tuned settings.
What is backpropagation?
The method that calculates, for every weight in a network, how a small change would affect the loss. It works backwards from the output through each layer, which is where the name comes from.
Related terms