How LLMs Are Made / Glossary

AI alignment

AI alignment is the work of making an AI system's behaviour match what people intend and value: helpful with everyday requests, honest about what it knows and careful to avoid harm.

A pretrained model copies patterns from the internet, the good and the bad. Alignment training adds written rules for how the model should behave and trains it on examples and ratings that follow them, using methods such as fine-tuning and RLHF.

The hard part is balance. A model that refuses too little can help cause harm, and one that refuses too much becomes useless: researchers built test sets to catch models refusing harmless questions. Some teams train separate reward models for helpfulness and for safety because the two can pull apart.

Other approaches have a model critique and revise its own answers against a written list of principles. Alignment is never finished; red teaming and monitoring after launch keep finding new gaps to fix.