How LLMs Are Made / Glossary

Temperature (AI)

Temperature is a setting that controls how predictable or varied a language model's output is. Low temperature makes it stick to the most likely tokens; high temperature makes less likely ones more common.

For each next token, the model scores every option and turns the scores into odds, then draws one. Temperature divides the scores before that step: below 1 the favourite gets even stronger, above 1 the odds even out.

Low temperature suits facts, code and data extraction, where you want the same reliable answer. Higher temperature suits brainstorming and creative writing, but pushed too far, the text becomes odd or makes less sense.

Temperature is often combined with other sampling rules, such as top-k, which draws only from the k most likely tokens, or top-p, which draws only from the smallest set of tokens that covers most of the odds.