Level 400: Engineering AI systems · Module 400.3

Two colleagues trace a page in an open binder beside plain file boxes on a café table.
Show me the paper trail

Data with a paper trail · Plain gear

Reviewed and published, version 1.0.0. Facts as of 18 Nov 2026; wording revised 19 Nov 2026. Not a credential. The research here comes from papers with dates. It is true of what those papers studied. Nothing here is legal advice. Where we did not check something, we say "not checked".

You will be able to:

Big Widget Company is made up for this course. So is everything about it here.

The service book

When you buy a used car, you want its service book. Who owned it? What was fixed, and when? Was it ever in a crash? The car might run fine today. The book is how you know what you are getting.

An AI is trained on data. The data is like the car's history: what was in it can shape how the AI behaves. So each set of data needs its own "service book".

Many people think: "This data is free on the internet, so we don't need to write anything down about it." But "free to look at" does not tell you why it was collected, what is in it, whether the people in it agreed, or what you are allowed to do with it.

In 2018, a group of researchers led by Timnit Gebru suggested a fix. In electronics, every part comes with a sheet that says how it works and how it was tested. They said every set of data should come with one too, and called it a datasheet for a dataset.1 It is meant for two groups: the people who made the data, so they think hard about how they made it, and the people who want to use it, so they can decide if it fits.1

A datasheet answers questions like these:1

Two tips from the researchers. First, data written by people counts as data about people.1 Notes typed by Big Widget's technicians count. Second, "we don't know" is an allowed answer. Leaving the whole thing blank is not.1

The researchers say to update the datasheet when data changes now and then. They admit that data which changes all the time is harder, and their method does not solve that.1 This course sets its own rule: every new version of the data gets an updated datasheet. It should say what was fixed and which version is the new one.1

One more course rule: the person who wrote the datasheet does not approve it. Someone else checks it.

The label on the AI

The AI trained on the data needs a label of its own. In 2019, researchers led by Margaret Mitchell called it a model card: a short paper, one or two pages, that goes with the AI.2 It says:2

The model card points back to the datasheet. Together they are the paper trail. This course adds one more line to the card: which AI it started from, if it was built on top of another one. The researchers also say the card should be updated with each new version of the AI.2 And they admit a limit: a card is only as honest as the people who wrote it.2

At Big Widget. The service team wants an AI that writes short summaries of technicians' repair notes. Their datasheet has two empty boxes: how the notes were collected, and the licence for some extra data they downloaded. Those two gaps stop the project. Nobody can check the notes match the job, and nobody knows what the team is allowed to do with the downloaded data.

If nobody knows the answer yet, the box says "not known yet", with the name of the person who will find out, and by when. That is honest. A blank box is not. Either way, the project waits until the answer is in.

Follow the trail back

Many people think: "The website I downloaded the data from says I can use it, so I can." Check first.

In 2023, a team of lawyers and AI researchers led by Shayne Longpre traced more than 1,800 sets of text data back to where they came from.3 On popular data-sharing websites, they found that over 70% of the licences were missing, and the error rate was over 50%.3 When they checked one big sharing site against what the original makers said, 66% of the licences were in a different group, often more generous than the makers meant.3 These numbers are from the 2023 version of their study.3

Why does this happen? Data gets copied and repackaged, again and again. Each copy can lose a rule or change one.3 It is like a recipe passed from friend to friend: by the fifth copy, the note "no nuts" might be gone.

So follow the trail, one step at a time:

  1. Where did the words or pictures first come from?
  2. Who put the data set together, and what licence did they give it?
  3. Where did you download it, and does that label match step 2?
  4. Which version did you train on?
  5. Which AI was trained on it? (The model card should say.)

Flag the trail when a step is missing or when two steps disagree.

At Big Widget. The download site says the extra data can be used for business. But the data's own paper says "not for business use". The trail disagrees with itself. When a data set had more than one licence, the researchers sorted it by the strictest one.3 That was how they grouped data sets for their study. It was not a ruling on which licence really counts. That is a question for a lawyer.

Big Widget's course rule borrows the caution. The answer is wait for a person. Do not use the data for a business product yet. Write down the problem. Ask the legal lead, who decides and writes down why. That rule is this course's, not legal advice.

Laws are starting to ask for this. One example: the European Union's AI law, as first passed in 2024, said that some high-risk AI systems must be trained on data handled with care. That includes keeping track of where the data came from, what personal data was first collected for, and what is missing from it.4 The law was changed in 2026, and this module has not checked how the change affects those rules.4 So treat it as an example, not as today's rule for you.

How bad data gets in

Five steps after a dataset is found to contain unlawful or poisoned material: stop use and hold, notify, remove from every copy, trace every affected model then retrain or retire it, and record the change in the datasheet and model cards.
The order is the course's procedure, built from its sources. Notify, as Carlini and colleagues did (s5); remove and check new downloads by hash (s4, s5); trace, then retrain or retire (s3, s4); update the datasheet and model cards (s1, s2).
Figure data as text

After bad data is found

Five steps in order. The order is the course’s procedure, built from its sources.

  1. HOLD — stop use and hold: freeze runs and releases; a named owner holds or constrains every model already in use. (course)
  2. NOTIFY — tell those affected: dataset maintainers, people policy and law name, and anyone given copies. [s5]
  3. REMOVE — clean every copy: including derived sets and caches; check new downloads match, for example by hash. [s4, s5]
  4. TRACE — retrain or retire: every model trained on it, and every model built on those. [s3, s4]
  5. RECORD — update the paperwork: datasheet erratum and new version; model card for every affected model. [s1, s2]

Objective: 400.3-c. Text equivalent of figures/after-bad-data-is-found.svg.

Many people think: "A few bad items in a giant pile of data can't matter." They can.

Illegal pictures. In 2023, researchers at Stanford checked a set of training data called LAION-5B. Parts of it had been used to train well-known AI picture-makers.5 Using computer tools that match known images, they found many hundreds of known images of child sexual abuse in it.5 This course does not describe them. The lesson is about the data: nobody had checked every picture, and illegal pictures were in a data set that AIs learned from. The researchers' advice covers the copies of the data, future data sets, the AIs already trained on it, and where those AIs are shared.5

Be careful what this means. It does not mean every picture ended up inside every AI. Only parts of it were used to train those picture-makers, and the study set out to measure how much the pictures may have played a part.5 What it does mean: every copy, and every AI trained on that data, has to be checked. You cannot just assume it is clean. "We don't know yet" is the honest answer until someone checks.

Planted data. Also in 2023, researchers led by Nicholas Carlini showed that an attacker could plant bad data on purpose.6 Many big data sets are not one fixed file. They are lists of web links, and each user downloads the pictures again later. If a website name (domain) in the list runs out and is not renewed, an attacker can buy it and put up a different picture.6 The researchers showed they could have planted bad data in 1 out of every 10,000 items of two big data sets, for just 60 US dollars.6 Other research has found that even tiny amounts of planted data can cause the AI to make chosen mistakes.6 To their knowledge, nobody had actually done this attack yet. It was a warning, not a crime report.6

So "too rare to matter" is wrong twice. Illegal is illegal, however rare. And planted data is put there on purpose, so a small, cheap amount can be enough.

Getting it out again

Deleting the bad items from one file is not enough. There are copies. And the AI has already learned from them.

Training an AI on data is expensive and very hard to undo.3 So this course plans for two fixes: train the AI again from the start on clean data, or stop using it. Training the same AI a bit more on clean data does not count. It still has what it learned before.

The researchers who planted the test data also suggested protections. One is a kind of fingerprint for each item. If a downloaded picture's fingerprint does not match the one recorded when the list was made, you know the file is different. You do not yet know why. Someone may have swapped it. Or it may have just been resized or saved in a new format, which also changes the fingerprint.6 So set it aside and find out. And a match only tells you the file is the same one that was recorded. It does not tell you that file was ever OK. After the researchers warned the people in charge, six of the ten data sets they studied started using these checks.6

The five steps when bad data is found. This order is this course's plan, built from the research above. It is not legal advice.

  1. Stop and hold. Stop training and stop releasing anything that uses the data. That does not stop an AI that is already in use. So for each AI that may be affected and is already running, even a copy at a supplier, a named person decides right away: take it out of use, or keep it running only within written limits. They write down which, and why. This course's default is to take it out of use. Only a named person can put it back. They also write down when they will look again. A hold that nobody comes back to ends as "no".
  2. Tell. Tell the people who look after the data, whoever your company's rules and the law say to tell, and anyone you gave a copy to. The Carlini team told the people in charge of each data set they studied.6
  3. Remove. Take the bad items out of every copy, and check new downloads match the clean version.56
  4. Find, then retrain or retire. Find every AI trained on the data, and every AI built on top of one of those. Train each again from the start on clean data, or stop using it.35
  5. Write it down. Update the datasheet and every model card, so everyone knows what changed.12

Keep notes from the very first step. "Write it down" comes last because it closes the job, not because you wait until the end.

At Big Widget. Every model card names the data version it was trained on, and the AI it started from. So the team can follow both trails and find the AIs that may be affected. If a card named only the data, the team would miss an AI built on top of an affected one. Any AI whose trail is not known yet stays on hold until someone finds out.

The three yeses

These are the data parts of the three yeses from module 100.9: Get to Yes · Stay at Yes · Recover to Yes. A full datasheet is evidence. It is not permission. A named person says yes. The AI does not, and neither does the tool that fills in the form. Each time, the person picks one of four answers: go ahead, only this far, wait for a person, or no.

What to share. A datasheet describes private data. It does not copy it. Keep customer names and note text out of it. The datasheets, notes and incident log stay in the team's own private records. If you keep a receipt of a decision, it holds no private content: what was decided, who decided, when, and which data and model versions. Never the notes, never names, and never the bad items or links to them. Here the receipt can be a plain note; it does not need to be signed. Modules 400.8 and 400.9 show how to make one a stranger can check.

My checklist:

Words to know: datasheet, model card, licence, paper trail, planted (poisoned) data, fingerprint (hash), hold, receipt.

Further learning (optional, not sources)

These free university materials go deeper. Some assume programming. They are pointers only; nothing here is copied from them.

Sources

  1. s1 Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford (2018). Datasheets for Datasets. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  2. s2 Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru (2019). Model Cards for Model Reporting. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩
  3. s3 Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, et al. (2024). A large-scale audit of dataset licensing and attribution in AI. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  4. s6 European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). Standard. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19; OJ text read via Publications Office Cellar DOC_1, EUR-Lex returned empty 202). ↩ ↩
  5. s4 David Thiel (author); Jeffrey Hancock (research team head), as the repository record lists them (2023). Identifying and Eliminating CSAM in Generative ML Training Data and Models. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19; abstract only, no PDF linked). ↩ ↩ ↩ ↩ ↩ ↩
  6. s5 Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, et al. (2023). Poisoning Web-Scale Training Datasets is Practical. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩

Next: the quiz