Skip to content
EN

Lesson 1 of 6 · 6 min · +100 XP

What is a large language model (LLM)?

A large language model (LLM) is an AI system trained on huge amounts of text to do one thing: predict the next piece of text. Given some words, it scores every possible next token, picks a likely one and repeats. Billions of numbers tuned during training store the patterns it learned, so it can answer, translate, summarize and write code.

▶ Try it

Why it matters

Every chatbot runs on an LLM. Once you see that it is a next-word predictor, it is clear why it sounds so fluent, why it knows so much and why it can still be confidently wrong.

In 6 minutes you'll be able to

  • When a chatbot answers, picture it ranking next words: that explains both its fluency and its slips.
  • Check names, dates and numbers it gives you, because a likely-sounding answer can be invented.
  • Use it for drafting, explaining and rephrasing, where likely wording is exactly what you want.

What you'll practise

  1. Large. Language. Model.

    Tap each card to unpack the name.

    📏 Large
    Billions of adjustable numbers, called parameters, trained on trillions of words. The size is a big part of why it works so well.
    💬 Language
    It works with text: reading it and writing it, in many human languages and in programming code.
    🧮 Model
    A giant mathematical function. Text goes in, and out comes a score for every possible next token.
    🎯 So what does it actually do?
    One thing, over and over: predict the next piece of text. Answering, summarizing and translating are all built on that.

  2. Take Vireo's seat

    Pick the word you think comes next. Then see how likely each option was. The percentages are illustrative.

    1. The cat sat on the ___

      • mat: 62% (most likely)
      • floor: 21%
      • sofa: 9%
      • moon: 0.1%
      • other words: 8%
      Vireo has seen this phrase so often that “mat” towers over the rest. A ranked list like this is all the model ever produces.

    2. Water freezes at zero degrees ___

      • Celsius: 80% (most likely)
      • Fahrenheit: 5%
      • today: 2%
      • other words: 13%
      A fact repeated in countless texts becomes a strong prediction. That's how a model “knows” things: as patterns, not as a list of facts.

    3. Translate into French: good morning → ___

      • bonjour: 88% (most likely)
      • bonsoir: 4%
      • hello: 2%
      • other words: 6%
      Nobody programmed a translator. Predicting text in many languages taught Vireo which words go together.

    4. Tomás, a training engineer at Vireo Labs, was born in ___

      • Spain: 20% (most likely)
      • 1990: 14%
      • Lisbon: 10%
      • Mexico: 9%
      • other words: 47%
      Vireo has never read about Tomás, yet it still ranks options and will pick one. That's how a fluent answer can be simply made up.

  3. True or myth?

    Sort each statement about large language models.

    • It learned from huge amounts of text → True
    • It looks answers up in a database → Myth
    • It writes one token at a time → True
    • Someone wrote a rule for every answer → Myth
    • It can sound sure and still be wrong → True
    • It gives the same answer every time → Myth

  4. The factory line

    This course follows Vireo from raw text to assistant. Put the stations in order.

    1. Collect huge amounts of text
    2. Clean it and cut it into tokens
    3. Pretrain: guess the next token, trillions of times
    4. Fine-tune it to follow instructions and feedback
    5. Test it hard, then release it

Remember

An LLM is a next-token predictor trained on vast amounts of text. Fluent means likely, not necessarily true.

An LLM predicts the next token, again and again. Its knowledge is patterns stored in billions of numbers.

Key facts

  • Most of today's large language models are built on the transformer, a design introduced in a 2017 research paper. · Source: Vaswani et al., Google, Attention Is All You Need, NeurIPS 2017
  • GPT-3 had 175 billion parameters, the adjustable numbers that store what a model learns. · Source: Brown et al., OpenAI, Language Models are Few-Shot Learners, 2020
  • Llama 3 was pretrained on over 15 trillion tokens of text. · Source: Meta, Introducing Meta Llama 3, 2024
  • People preferred answers from a 1.3-billion-parameter model fine-tuned to follow instructions over answers from the 175-billion-parameter GPT-3: training after pretraining matters. · Source: Ouyang et al., OpenAI, Training language models to follow instructions with human feedback, 2022

Updated:

Questions people ask

What does LLM stand for?

LLM stands for large language model: an AI model with billions of parameters, trained on huge amounts of text to predict the next token.

Is a chatbot the same as an LLM?

A chatbot is an app built around an LLM. The model predicts text; the app adds the chat window, extra training to follow instructions, safety rules and sometimes tools like web search.

Does a large language model understand what it says?

Not the way people do. It has learned very detailed patterns of language, which makes it useful, but it has no built-in way to check whether a sentence is true.

Why are they called large?

Because of their size: billions of parameters, trained on trillions of words with enormous computing power. Bigger models trained on more data have generally become more capable.

Related terms