Level 100: How AI works: AI at work and in life · Module 100.3

Two coworkers talk while one arranges paper slips in a row on the table.
One piece at a time.

How it writes · Plain gear

You will be able to: say how an AI like GPT-3 writes, one small piece at a time; say why an AI can sound sure and still be wrong; and name one way to check.

Laying track one piece at a time

Read the request and text so far, estimate the next token, choose and append one, and repeat with the new context.
The model reads what is in context, estimates how likely each next token is, adds a chosen token and repeats with that piece in its context (s1).
Figure data as text

One piece at a time

A loop: step 3 leads back to step 1.

  1. Read the context: the request and the text written so far. [s1]
  2. Work out how likely each possible piece is to come next. [s1]
  3. Choose and add one token at the end. [s1]
  4. Repeat from step 1 with the new token in the context. [s1]

Objective: 100.3-a. Text equivalent of figures/next-token-loop.svg.

Picture a toy train. There is no finished track. As the train rolls, one new piece of track is laid in front of it. Then the next. Then the next. Each piece is chosen to fit the pieces already down.

Some AIs write a bit like that. This lesson is about that kind of AI. GPT-3 is one example.1

It cuts writing into small pieces called tokens. In the design we look at here, a token can be a whole word or part of a word.2

Then it writes one token at a time. Each time, it looks at the text it has in front of it: your question, plus what it has already written. It works out how likely each possible token is to come next. It picks one and adds it. Then it does the same thing again.2 Experts have a long name for this: autoregressive.1

There is a limit to how much text it can look at once. GPT-3 could look at about 2,000 tokens at a time.1 Text outside that limit is not part of what it looks at.

So the words are made one small piece at a time. Each piece it has written becomes part of what it looks at for the next piece.2

Here is the key point. It picks the likely next piece. Likely is not the same as true. A US government standards group, NIST, says the same thing. These AIs predict the next word, and that can give you true words or false ones.3 So picking a likely piece is not checking the facts.

What we do not cover here. This lesson is about how the words are made. It is not about how an app shows them to you, all at once or bit by bit. We have not checked any app. And it does not say what goes on inside the AI while it picks. That is a harder question.

Sounding sure is not being right

AI writing can sound very real. In one test, people read short news stories. Some were written by people and some by an AI. People had to guess which was which. They were right only a little more often than if they had tossed a coin.1

That test was about who wrote it, not whether it was true. But it shows how good the writing can be. So smooth writing does not prove the facts are right.

Sometimes an AI makes things up. It might invent a book that does not exist, a quote nobody said or a number from nowhere. And it says it in the same calm, sure voice. People call this a hallucination. NIST calls it confabulation: false content stated with confidence, which can fool people.3

It is not just a glitch in a broken AI. NIST says it is a natural result of how these AIs are built.3 So an AI that is working normally can still make things up. How often depends on the AI and the task. Here are two older tests:

Those numbers are about those AIs, on those tasks, back then. They are not about the AI you use today.

Here is the sneaky part. The made-up bit can look like proof. An AI can add sources or reasons that make its answer look trustworthy, even when they are invented.3 The trouble starts when someone believes it, often because it sounds so sure, and acts on it.3

A real case: court cases that did not exist

This really happened. In 2023, a US court looked at a case called Mata v. Avianca. The court found that lawyers sent in court opinions that did not exist, "with fake quotes and citations created by the artificial intelligence tool ChatGPT."6 When the judge asked about them, they kept standing by them.6 And one lawyer signed his papers without reading any of the cases he cited.6

The fake cases looked real. They had names and quotes. They were said to be in well-known law books and a legal database.6

People did try to look them up. Lawyers for the other side said they could not find most of them.6 The court searched too, and could not find several of them.6 And the lawyer who wrote the papers told the judge he had looked up one of the cases and could not find it. He had cited it anyway.6

So "look it up" was not enough on its own. The rule this course takes from the story is: if you cannot find it, stop. Do not use it until you find the real thing.

This story does not tell us how the AI made the fake cases, or how often AIs do this. It shows what happens when a check fails and the work goes ahead anyway.

The AiGovOps Foundation wrote about this case in a blog post by Bob Rapp (November 2026). The post is an editorial summary. It gives the Foundation's rule: "every consequential AI output is independently verified against the system of record before it is used or filed."7 That means: anything that matters gets checked against the real record before you use it. That is the Foundation's idea, not what the court said.

One way to check: open the source

NIST advises organisations that run AI systems to review and verify the sources and citations in what the AI writes, both before the system goes live and while it is in use.3 This course turns that into three steps you can take yourself:

  1. Open every source. Find the real page, report or book. If you cannot find it, it does not count.
  2. Find the exact words. Make sure the source really says what the AI says it does. If no real source backs a claim, take the claim out.
  3. Ask a real person when it matters, like a report, a customer or a big decision. Asking the same AI "are you sure?" does not count. The Foundation's post puts it this way: "a second question to the same model is not a check."7

Stuck? It is fine to say "I don't know yet." Write "not checked" and ask someone. The Foundation's practice site says "I don't know yet" is always an answer, and it always pairs you with someone.8

Some AI tools look up documents first and then write. In one 2020 study, an AI that did this wrote quiz questions that people judged more factual than ones from an AI that did not.9 That is one test on one kind of task. More factual is not perfect, so you still check each claim.

At Big Widget Company

Big Widget Company is made up, and so are its people.

Luis buys parts. He asks an AI why parts cost more last quarter, and asks for sources.

The answer sounded just as sure both times. It was written one likely piece at a time. The only thing that caught the fake sources was Luis.

The three yeses. The Foundation uses three short phrases: Get to Yes, Stay at Yes and Recover to Yes.10 Here is the source-checking part of each.

Keep notes, not secrets. Luis writes down which sources he opened and what he found. He keeps that in his team's own files. The Foundation's receipt is different. It never carries "prompts, outputs, payloads, personal data".8 So the AI's answer and the purchasing data do not go in it.

Words to know: token, hallucination, confabulation, source.

Sources

  1. s2 Tom B. Brown et al. (2020). Language Models are Few-Shot Learners. Paper. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩ ↩ ↩
  2. s1 Ashish Vaswani et al. (2017). Attention Is All You Need. Paper. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩ ↩
  3. s3 Chloe Autio et al. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. Standard. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩
  4. s5 Stephanie Lin, Jacob Hilton, and Owain Evans (2021). TruthfulQA: Measuring How Models Mimic Human Falsehoods. Paper. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩
  5. s4 Long Ouyang et al. (2022). Training language models to follow instructions with human feedback. Paper. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩
  6. s7 United States District Court (court-filed opinion; RECAP copy hosted by CourtListener) (2023). Mata v. Avianca, No. 22-cv-1461 (PKC), Opinion and Order on Sanctions (docket entry 54). Record. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩
  7. s8 Bob Rapp (AiGovOps Foundation) (2026). Asking the bot to check the bot — Mata v. Avianca, and the citations nobody opened. Official doc. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩
  8. s10 AiGovOps Foundation (n.d.). AiGovOps Foundation Practice (home page). Official doc. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩
  9. s6 Patrick Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Paper. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩
  10. s9 AiGovOps Foundation (n.d.). AiGovOps Foundation Manifesto. Official doc. Checked 2026-11-17 by source-checker agent (Claude, separate round-3 re-check session; did not write or revise the module), 17 Nov 2026. ↩ ↩

Next: the quiz