Skip to content
EN

Lesson 6 of 6 · 6 min · +100 XP

Is AI trained on my data, and can I opt out?

Language models are trained mostly on public text, which can include your posts, reviews and articles. Public doesn't mean free to use: copyright, licences and consent all matter. Website owners can block AI crawlers in a file called robots.txt, and some platforms offer opt-out settings. Opting out stops future collection but doesn't remove text from models already trained.

▶ Try it

Why it matters

Behind every training sentence is a person who wrote it. Knowing the rules helps you judge AI makers, and protect your own writing if you want to.

In 6 minutes you'll be able to

  • If you run a website, you can block AI crawlers in robots.txt while staying visible in search.
  • Check the AI or privacy settings of platforms you post on; some let you opt out of training.
  • Remember that opting out works going forward: it doesn't remove text from models already trained.

What you'll practise

  1. Four words that decide

    Tap each card to see what's behind it.

    ©️ Copyright
    Whoever writes a text usually owns it automatically. Copying it can need permission, and whether AI training counts is being argued in courts.
    📜 Licence
    Permission to use a work on stated terms. Some texts are openly licensed for reuse; others are licensed to AI makers for money.
    🤖 robots.txt
    A small file on a website that tells crawlers which pages they may visit. Site owners can use it to say no to AI crawlers.
    🧠 Can text be taken back out?
    Not easily. Opting out stops future collection, but what a model already learned is spread across billions of numbers.

  2. Run Vireo's crawler

    Vireo's crawler asks before each site. Let it crawl, check the details first, or skip the site.

    Collect text for Vireo's next training run, the fair way.

    1. Copy a library of old novels whose copyright has run out
      Tool Fetch · whole site · Found via Vireo's crawl list · Site notice: every book here is in the public domain and free to reuse.

      → Crawl: Public-domain texts are free for anyone to use. A great source, so let it run.

    2. Fetch every recipe from Rosa's baking blog
      Tool Fetch · all pages · Found via A link on another site · robots.txt: VireoBot is not allowed on any page.

      → Skip: Rosa's robots.txt says no to Vireo's crawler. A responsible crawler respects that and moves on.

    3. Download a news archive by getting round its paywall
      Tool Fetch · bypass login · Found via Its own idea · Articles behind a subscription, reached by skipping the login page.

      → Skip: Getting round a paywall takes work that's sold for money. That needs a licence, not a trick.

    4. Collect answers from a public help forum
      Tool Fetch · public pages · Found via Vireo's crawl list · Forum terms: posts are shared under an open licence that allows reuse with credit.

      → Check first: Public isn't automatically free to use, so check the terms first. Here they allow reuse.

    5. Copy a novelist's sample chapters marked “not for AI training”
      Tool Fetch · all pages · Found via Its own idea · Footer: © the author. Not to be used for AI training.

      → Skip: The author has clearly said no. Respecting a stated opt-out is part of crawling fairly.

    6. Add a publisher's textbooks under a new licence deal
      Tool Licence · paid agreement · Found via Vireo's legal team · Contract: Vireo pays the publisher, and authors are told and share the fee.

      → Check first: A licence can make it fair, but read the terms first: who agreed, and what do they get?

  3. Protect Rosa's blog

    Rosa wants readers and search engines, but not AI training. Set up her blog.

    • Block AI training crawlers in robots.txt
    • Block all search engines too
    • Add a notice: “Not for AI training”
    • Show her home address on the About page
    • Check AI settings on the platforms she posts on

Remember

Public text still belongs to someone. Fair training respects copyright, licences and opt-outs.

Public isn't the same as free to use. Responsible crawlers respect copyright, licences and opt-outs.

Key facts

  • The robots.txt rules that tell crawlers where they may go were formalised as an internet standard in 2022. · Source: IETF, RFC 9309: Robots Exclusion Protocol, 2022
  • In the EU, rights holders may reserve their works from text and data mining in a machine-readable way, for example in robots.txt. · Source: EU Directive 2019/790 on Copyright in the Digital Single Market, Article 4
  • The EU AI Act requires providers of general-purpose AI models to have a policy for respecting EU copyright law and to publish a summary of the content used for training. · Source: EU Artificial Intelligence Act, Regulation (EU) 2024/1689, Article 53
  • Researchers found that websites restricted AI crawlers in their robots.txt much more often in 2023 and 2024 than before. · Source: Longpre et al., Data Provenance Initiative, Consent in Crisis, 2024

Updated:

Questions people ask

Can AI companies use my public posts for training?

Often they do, if the posts are publicly reachable. Whether that is allowed depends on copyright law in your country, the platform's terms and whether you or the site opted out.

How do I stop AI from training on my website?

Add rules to your site's robots.txt that block the crawlers AI companies use for training. Most publish their crawler names. You can block them and still allow search engines.

Can I remove my data from an AI model that's already trained?

Usually not directly. What a model learned is spread across its parameters. Some companies accept requests to exclude your data from future training or to stop a model repeating personal details.

Is training AI on copyrighted text legal?

It depends on the country and is still being decided. Some laws allow text and data mining unless the owner opts out, and several lawsuits are testing where the lines are.

Related terms