How LLMs Are Made / Glossary

Fine-tuning

Fine-tuning means training an already pretrained model a little more on a smaller, focused set of examples, so it gets better at a task or behaves in a particular way.

Pretraining gives a model broad knowledge; fine-tuning shapes it. Instruction tuning, also called supervised fine-tuning, trains a base model on requests paired with ideal answers written by people, which turns a text-continuer into an assistant.

Fine-tuning needs far less data than pretraining. Fine-tuning a model on many tasks written as instructions made it better at new tasks it was never tuned on, and one study got strong assistant behaviour from just 1,000 carefully chosen examples.

Companies also fine-tune models on their own data, for example to match a writing style or a specialist field. Lighter methods such as LoRA train only a small set of extra weights, so fine-tuning can run on far less hardware.