How do AI reasoning models learn to think step by step?
Reasoning models are trained with reinforcement learning on problems that have checkable answers, like maths and code. The model writes many attempts, each a chain of thinking steps, and an automatic checker rewards only the ones that reach the right answer. Over time, careful habits such as working step by step and checking its own work get reinforced.
▶ Try it
Why it matters
Some answers can be checked automatically, with no human needed. That lets a model practise on millions of problems and learn which ways of thinking actually reach the right answer.
In 6 minutes you'll be able to
- Maths and code answers can be checked automatically, so a model can practise on millions of problems.
- Reinforcement learning rewards the chains of thought that reach the right answer.
- Habits like breaking problems down and checking work grow because they pay off. Still check answers that matter.
What you'll practise
Check the chains
For each attempt, tap the first wrong step, or “No mistake”. Then run the checker and watch the bars.
Problem: 17 × 24 = ? → 408
- Blurts an answer: Looks like about 400 → [✗ So: 398] → 398 ✗
- Steps, no check: 17 × 20 = 340 → [✗ 17 × 4 = 58] → 340 + 58 = 398 → 398 ✗
- Steps, then checks: 17 × 20 = 340 → 17 × 4 = 68 → 340 + 68 = 408 → Check: 17 × 25 − 17 = 408 ✓ → 408 ✓
Problem: A $40 shirt is 25% off. What do you pay? → $30
- Steps, no check: 25% of 40 = 10 → 40 − 10 = 30 → $30 ✓
- Blurts an answer: [✗ 25% off, so $25] → $25 ✗
- Steps, then checks: 25% = a quarter → 40 ÷ 4 = 10 → 40 − 10 = 30 → Check: 30 is 75% of 40 ✓ → $30 ✓
Problem: Mia is 3 years older than Leo. Together they are 21. How old is Mia? → 12
- Steps, then checks: Leo = x, Mia = x + 3 → 2x + 3 = 21 → x = 9, so Mia = 12 → Check: 12 + 9 = 21 ✓ → 12 ✓
- Steps, no check: Leo = x, Mia = x + 3 → 2x + 3 = 21 → x = 9 → [✗ So Mia is 9] → 9 ✗
- Blurts an answer: About half of 21 → [✗ So Mia is 10] → 10 ✗
Problem: A train leaves at 9:47 and the trip takes 2 h 38 min. When does it arrive?
9:47 + 2 h = 11:47 → 11:47 + 38 min = 11:85 → Wait: 85 min isn't a time. 85 min = 1 h 25 min → 11:00 + 1 h 25 min = 12:25 → Check: 9:47 → 12:25 is 2 h 38 min ✓ → 12:25 ✓Can a checker score it?
This training needs answers a program can check. Which tasks qualify?
- Solve for x: 3x + 5 = 20 → A checker can score it
- Write a kind note to a new neighbour → Needs human judgment
- Fix this code so all its tests pass → A checker can score it
- Is this explanation clear for a 12-year-old? → Needs human judgment
- How many days are there from 3 March to 1 April? → A checker can score it
- Which of two poems is more moving? → Needs human judgment
Which habits got rewarded?
Nobody programmed Vireo's thinking style. Which habits does rewarding right answers tend to grow? Pick all that apply.
- Breaking a problem into small steps
- Checking the answer a second way
- Trying another approach after a dead end
- Thinking longer on harder problems
- Sounding more confident
- Using longer words
Remember
Reasoning models practise on problems with checkable answers. Rewarding right answers makes careful, step-by-step thinking more likely.
Reward the right answer often enough and good thinking habits grow on their own.
Key facts
- Simply prompting a large model to write out its working step by step, called chain-of-thought prompting, was shown to improve its accuracy on maths word problems. · Source: Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Google, 2022
- OpenAI reported that its o1 model performs better both with more reinforcement-learning training and with more time spent thinking before it answers. · Source: OpenAI, Learning to Reason with LLMs, 2024
- DeepSeek reported that reinforcement learning with simple rule-based rewards for correct answers led its R1-Zero model to develop long reasoning and self-checking without human-written reasoning examples. · Source: DeepSeek-AI, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, 2025
- Tasks with verifiable answers, like maths results or code that must pass tests, suit this training because the reward can be computed automatically, without a human rater.
Updated:
Questions people ask
What is a reasoning model?
A reasoning model is a language model trained to work through a problem in a chain of thinking steps before it answers. It's usually trained with reinforcement learning on problems whose answers can be checked.
What is chain of thought?
Chain of thought is the step-by-step working a model writes out on the way to an answer, like showing your work in maths. It often makes answers to multi-step problems more accurate.
Why do reasoning models take longer to answer?
They write many thinking steps before the final answer, and harder problems get more steps. Each step takes time and computing power, but it tends to improve accuracy.
Do reasoning models still make mistakes?
Yes. They make fewer slips on checkable problems, but they can still reason from a wrong fact or produce a convincing yet flawed chain. Check answers that matter.
Related terms