Skip to content
EN

Lesson 3 of 8 · 6 min · +100 XP

How do large language models (LLMs) write their answers?

An AI chatbot runs on a large language model (LLM) that writes one small word piece, called a token, at a time. For each step it scores every possible next token by how likely it is after the text so far, picks one, adds it and repeats. Because it predicts likely words rather than looking up facts, check important answers.

▶ Try it

Why it matters

A chatbot doesn't look answers up. It predicts the next token again and again, which explains why it sounds fluent, why answers vary and why it can be wrong.

In 6 minutes you'll be able to

  • Chatbots write one token at a time by picking a likely next piece of text.
  • Temperature makes answers safer or more surprising, so replies can vary.
  • A confident, well-written answer can still be wrong. Check what matters.

What you'll practise

  1. Inside the engine room

    Tap each card to see what's behind it.

    🧩 What is a token?
    A small piece of text: a whole word, part of a word or a punctuation mark. “Unbelievable” might be split into “un”, “believ” and “able”.
    📚 How did Lumi learn?
    By reading huge amounts of text and practising one game: hide the next token, guess it, compare, adjust. Billions of times.
    📊 What does it predict?
    Not one word, but a score for every possible next token: how likely each one is after the text so far.
    🔁 So how does a reply appear?
    Lumi picks a likely token, adds it to the text and predicts again. Word by word, until the answer is done.

  2. Be the language model

    Pick the next word. Then see how likely each option was. The percentages are illustrative, made up to show the idea, not taken from a real model.

    1. Thank you so much for your ___

      • help: 40% (most likely)
      • time: 15%
      • message: 8%
      • tractor: 0.1%
      • other words: 37%
      After “thank you so much for your”, words like “help” appear very often in human writing, so they get high scores. “Tractor” is possible, just very rare.

    2. The capital of France is ___

      • Paris: 85% (most likely)
      • a: 5%
      • Lyon: 1%
      • other words: 9%
      When a fact appears in countless texts, one word towers over the rest. Here the most likely word also happens to be true.

    3. Once upon a time, there was a ___

      • little: 30% (most likely)
      • young: 15%
      • dragon: 5%
      • toaster: 0.2%
      • other words: 50%
      Chatbots don't always take the top word. A setting called temperature controls how adventurous the pick is. Try the switch.
      • Temperature: Low. Once upon a time, there was a little girl who lived in a small village.
      • Temperature: High. Once upon a time, there was a toaster who dreamed of visiting the moon.

    4. Dr Mira Okafor, an invented scientist, won the Nobel Prize in ___

      • Physics: 30% (most likely)
      • Chemistry: 25%
      • Medicine: 20%
      • nothing: 2%
      • other words: 23%
      Dr Okafor doesn't exist, yet every option sounds plausible and something will be picked. That's how a chatbot can produce a fluent answer that is simply false.

  3. From question to answer

    You ask Lumi a question. Put the steps in order.

    1. You send your message
    2. The text is split into tokens
    3. The model scores every possible next token
    4. One likely token is picked and added to the text
    5. The model predicts again, until the answer is finished

  4. What does that mean for you?

    1. You ask Lumi the same question twice and get two different answers. Why?

      • It picks among likely words, with some randomness (Answer)
      • It changed its mind after thinking it over

    2. Lumi answers smoothly and confidently. What can you conclude?

      • It's well-written, which says nothing about it being true (Answer)
      • It must be correct, it sounds so sure

Remember

A chatbot predicts the next token again and again. Fluent means likely, not necessarily true.

A chatbot is a next-word predictor. Fluent means likely, not necessarily true.

Key facts

  • In 1951, Claude Shannon measured how predictable English is by asking people to guess the next letter of a text, an early version of the game language models play today. · Source: C. E. Shannon, Prediction and Entropy of Printed English, Bell System Technical Journal, 1951
  • Most of today's large language models are built on the Transformer, a neural network design introduced in a 2017 research paper. · Source: Vaswani et al., Attention Is All You Need, NeurIPS 2017
  • Under the EU AI Act, providers of AI systems that talk directly with people, such as chatbots, must make sure people know they are interacting with AI, unless that is already obvious. This applies from 2 August 2026. · Source: EU Artificial Intelligence Act, Regulation (EU) 2024/1689, Article 50
  • A language model doesn't store a list of facts to look up. What it knows is spread across the numbers in its model, learned from patterns in its training text.

Updated:

Questions people ask

What is a large language model?

A large language model (LLM) is an AI model trained on huge amounts of text to predict the next token. Chatbots use an LLM to write replies one piece at a time.

What is a token in AI?

A token is the small piece of text a language model works with: a whole word, part of a word or a punctuation mark. Models read and write text as sequences of tokens.

Does a chatbot understand what it says?

Not the way people do. It has learned very detailed patterns of language, which lets it produce useful answers, but it has no direct way to know whether a sentence is true.

Why do AI chatbots give different answers to the same question?

Chatbots usually sample from several likely next words instead of always taking the top one. A different early choice leads to a differently worded, or sometimes different, answer.

Related terms