How does AI learn from examples?
AI learns through machine learning: a computer is shown many labelled examples, such as photos tagged “cat” or “dog”. During training it adjusts a model until its guesses match the labels, then it is tested on new examples it has never seen. The more varied the training data, the better the model handles real situations it wasn't shown.
▶ Try it
Why it matters
A model only knows what its examples showed it. Choosing varied, well-labelled training data matters more than piling up more of the same.
In 6 minutes you'll be able to
- Machine learning = labelled examples + training + testing on new data.
- Varied training data helps more than lots of the same kind.
- Wrong labels and missing examples turn into wrong answers.
What you'll practise
The recipe
Every machine-learning model is made the same way.
- 🗂️ Training data
- A collection of examples, like thousands of pet photos. This is everything the model will ever know.
- 🏷️ Labels
- The right answer for each example: “cat” or “dog”. Usually added by people, so mistakes in labels become mistakes in the model.
- ⚙️ Training
- The model guesses, is compared with the label and adjusts itself, over and over, until its guesses mostly match.
- 🧪 Testing
- Finally it's tested on new examples it never saw. That shows whether it learned the pattern or just memorised the photos.
Train a tiny model
Pick up to 5 training photos, then press Train. The model will be tested on 5 new photos.
Cat · Dog
Training examples
- Orange tabby cat on a sofa → Cat
- Another orange tabby, asleep → Cat
- Black cat in a dim room → Cat
- Tiny kitten with big eyes → Cat
- Grey cat seen from behind → Cat
- Big golden dog on the grass → Dog
- Another big golden dog, running → Dog
- Small fluffy dog on a lap → Dog
- Puppy chewing a shoe → Dog
- Dog standing in the snow → Dog
Test items
- A black cat at night → Cat
No dark cat in the training set, so the model never learned that cats can look like this. - A kitten in a basket → Cat
Kittens look different from grown cats. Without a kitten example, the model had nothing to go on. - A big dog in a park → Dog
The model never saw a large dog, so it didn't connect this shape with “dog”. - A tiny dog in a handbag → Dog
Small dogs are easy to confuse with cats if the model only saw big ones. - A husky in deep snow → Dog
A snowy background was new to the model. Varied settings matter, not just varied animals.
What did you learn?
Noor can add 1,000 more photos. Which set will help the model most?
- 1,000 varied photos: all sizes, colours, ages and places (Answer)
- 1,000 more photos of orange tabby cats
The model scores 100% on its training photos but poorly on new ones. What happened?
- It memorised the examples instead of learning the pattern (Answer)
- The new photos must be labelled wrong
A helper labelled some dogs as “cat” by mistake. What does that do?
- The model learns those mistakes too (Answer)
- Nothing, the model will spot the errors
Remember
A model is only as good as its examples. Variety beats volume.
Key facts
- In a 1959 paper, computer scientist Arthur Samuel described a checkers program that learned to play better than the person who wrote it, an early landmark of machine learning. · Source: A. L. Samuel, Some Studies in Machine Learning Using the Game of Checkers, IBM Journal of Research and Development, 1959
- The ImageNet project, whose labelled photos helped launch the modern deep-learning boom, described 3.2 million images labelled by online crowd workers in its first paper. · Source: Deng et al., ImageNet: A Large-Scale Hierarchical Image Database, CVPR 2009
- A classic handwriting dataset, MNIST, splits its images into 60,000 for training and 10,000 for testing, so models are judged on digits they have never seen. · Source: LeCun, Bottou, Bengio & Haffner, Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, 1998
- Testing a model on examples it never saw during training is the standard way to check that it learned a general pattern rather than memorising its training data.
Updated:
Questions people ask
What is training data?
Training data is the set of examples a machine-learning model learns from, such as labelled photos, texts or recordings. The model can only learn patterns that appear in this data.
What is the difference between supervised and unsupervised learning?
In supervised learning every example comes with the right answer, a label, like “cat”. In unsupervised learning there are no labels, and the model looks for groups or structure in the data by itself.
How much data does machine learning need?
It depends on the task. Simple tasks can work with hundreds of examples, while large models use billions. Variety and correct labels often matter as much as the amount.
What is overfitting?
Overfitting is when a model memorises its training examples instead of learning the general pattern. It looks perfect in training but performs poorly on new data.
Related terms