All topics / How LLMs Are Made
How a large language model is made, from raw text to chatbot
Follow one model through the whole factory: collecting and cleaning text, cutting it into tokens, months of next-token practice on thousands of chips, then the training that turns an autocomplete into a helpful assistant. Each lesson is a 6-minute animated drawing with a quick game. No maths, no code.
- 24 lessons
- 144 min
- Free
- No sign-up
Hi, I'm Tomás. I help train language models at Vireo Labs. I'll walk you through every station of the line, from a messy pile of web pages to a model that answers questions, and show you where the hard choices are made.
The raw material
Where the text comes from, how it is cleaned and how it becomes numbers
- 1
What is a large language model?
A giant next-word predictor, built from text
- 2
Where the training text comes from
Web crawls, books, code and conversations
- 3
Cleaning the data
Removing duplicates, spam and private details
- 4
Tokens: how text becomes pieces
Why a model reads "unbelievable" as three parts
- 5
Embeddings: words as points in space
Similar meanings end up close together
- 6
Whose data is it?
Copyright, consent and opting out
Pretraining
The transformer, attention and months of guessing the next token
- 7
The transformer
The layered design behind every modern LLM
- 8
Attention
How each word looks back at the words that matter
- 9
Guess the next token
The one training game, played trillions of times
- 10
Learning from mistakes
Loss, nudges and billions of tiny dials
- 11
Scaling up
More data, more parameters, more chips
- 12
The base model
A brilliant autocomplete that is not an assistant yet
Becoming an assistant
Instructions, human feedback, reasoning, safety and tools
- 13
Instruction tuning
Teaching the model to follow requests
- 14
Learning from human feedback
People compare two answers and pick the better one
- 15
Reward models
A second model that scores answers
- 16
Learning to reason
Thinking step by step on problems with checkable answers
- 17
Safety training
Refusing harm, staying honest, admitting doubt
- 18
Tools and agents
Search, code and actions in the world
Testing and shipping
Benchmarks, red teams, smaller models and how a reply is generated
- 19
Benchmarks and evals
How a new model gets its report card
- 20
Red teaming
Experts try to break the model before launch
- 21
Making models smaller
Distillation and quantization
- 22
How a reply is generated
One token at a time, with a dash of chance
- 23
Context windows and memory
What the model can see at once, and what it forgets
- 24
Model cards and release
What a responsible launch tells you
How it works
- 01
See it
An animated drawing shows how the thing really works.
- 02
Try it
A short game makes you use it: spot, sort, switch, decide.
- 03
Keep it
Three takeaways and a badge for your sheet.
Questions people ask
Do I need to know maths or programming?
No. Every step is shown as an animated drawing with everyday examples. You will see what a token, a weight or a reward model is without a single formula.
How long does it take to train a large language model?
Pretraining a frontier model usually runs for weeks to months on thousands of specialised chips, after a long phase of collecting and cleaning data. The fine-tuning that turns it into an assistant comes on top.
Is this course free?
Yes. Every lesson is free, with no sign-up and no paywall.
How is this different from the AI Literacy course?
AI Literacy is about understanding and using AI well. This course goes behind the scenes and follows one model through every stage of how it is built, tested and released.