
Data with a paper trail · Standard gear
Reviewed and published, version 1.0.0. Facts as of 18 Nov 2026; wording revised 19 Nov 2026. Not a credential. Research findings are dated and true of what those studies tested. The legal text is quoted as a dated, historical example, not as advice on what the law requires of you today. Where we did not check something, we say "not checked".
You will be able to:
- write a datasheet for a training or evaluation dataset (motivation, composition, collection, consent, licence, known gaps) and a model card for the model trained on it;
- trace a dataset's provenance and licence from its source to the model, and flag where the chain is missing or contradicts itself;
- explain how harmful, unlawful or poisoned content in a web-scale dataset reaches the models trained on it, and what removal requires.
Big Widget Company and everything about it in this module are made up for teaching.
1. Every dataset needs a datasheet
A common belief: "The data is public, so documenting it is optional." Public says who can see the data. It says nothing about why it was collected, what is in it, whether the people in it agreed, or what you may do with it. A model trained on the data inherits those open questions.
In 2018 Timnit Gebru and colleagues proposed datasheets for datasets. Their analogy comes from electronics: there, "every component, no matter how simple or complex, is accompanied with a datasheet" describing how it behaves, how it was tested and how it should be used. They proposed that every dataset carry one too, documenting "its motivation, composition, collection process, recommended uses, and so on".1 A datasheet serves two groups: the people who create a dataset, who are pushed to reflect on how they made it, and the people who consume it, who need enough information to decide whether to use it.1
The paper's questions are grouped by the stages of a dataset's life: motivation, composition, collection process, preprocessing/cleaning/labelling, uses, distribution and maintenance.1 For this module, six parts matter most:
| Part | Questions the datasheet answers (from the paper) |
|---|---|
| Motivation | For what purpose was the dataset created? Who created it, and who funded it?1 |
| Composition | What does each instance represent, and how many are there? Is it a sample of a larger set, and is the sample representative? Are there errors, noise or redundancies? Does it rely on external resources, such as websites, that may change or carry their own restrictions? Does it contain confidential or sensitive data?1 |
| Collection | How was the data acquired, by whom, over what timeframe, and was there an ethical review?1 |
| Consent | For data about people: were they notified? Did they consent to the collection and use of their data, and can they revoke that consent?1 |
| Licence | Will the dataset be distributed under a copyright or other IP licence, or terms of use? Have third parties imposed IP-based or other restrictions on the data?1 |
| Known gaps | Is any information missing from instances? Is there anything about its composition or collection that might affect future uses, and are there tasks it should not be used for?1 |
In the paper, consent sits in the collection section and licence in the distribution section; this course pulls them out because they are the two answers that most often block a release. Three details from the paper shape how you write one:
- Data about people is broad. The authors recommend a broad reading: "any dataset containing text that was written by people relates to people".1 A service-notes dataset written by technicians relates to people.
- "Unknown" is an allowed answer; skipping is not. In the paper's example datasheet, written for a dataset the authors did not create, some answers read "Unknown to the authors of the datasheet." They recommend answering as many questions as possible "rather than skipping the datasheet creation process entirely".1
- It is not a form to auto-fill. The authors say the process "is not intended to be automated", because automation would defeat the purpose of making creators reflect.1 They also say datasheets "do not provide a complete solution" to bias or harm.1 A filled-in box is a claim, not a clearance: "consent: yes" means little until someone can show where that answer came from.
A datasheet changes when the data changes. The maintenance questions ask whether there is an erratum, whether the dataset will be updated (for example to delete instances), and how older versions will be supported or retired.1 The authors recommend updated datasheets for datasets that change only infrequently, and they say plainly that their questions and workflow may pose problems for datasets that change all the time.1 This course goes further, as its own rule: every new version of a dataset gets an updated datasheet with a new version number, however often that happens. If your data changes too often for that, say so in the datasheet, and name who decides how it is documented.
The writer is not the checker. Also this course's rule: the person who wrote a datasheet does not approve it for release. A second person reads it against the data and the licence trail.
2. The model card travels with the model
A datasheet describes data. The model trained on it needs its own document. In 2019 Margaret Mitchell and colleagues proposed model cards: short records, one to two pages, that accompany a trained model. They describe model cards as "complements" to datasheets: datasheets cover the data going in, model cards the trained model's characteristics.2
The paper proposes these sections:2
- Model details: who developed it, date, version, type, training approach, licence, where to send questions.
- Intended use: primary uses, primary users, and out-of-scope uses.
- Factors: the groups, instruments and environments across which performance may vary.
- Metrics: which measures, which decision thresholds, and how uncertainty was estimated (for example confidence intervals).
- Evaluation data: which datasets, why, and how they were preprocessed.
- Training data: ideally as much as for evaluation data; if that is not possible, at least "details of the distribution over various factors in the training datasets".
- Quantitative analyses: results broken down by factor, alone and in combination.
- Ethical considerations.
- Caveats and recommendations.
Three points connect the card back to the datasheet:
- The evaluation data section should ideally point to documents "that provide visibility into the source and composition of the dataset".2 At Big Widget that means the datasheet, by name and version.
- When training data is proprietary or under a non-disclosure agreement, the authors still ask for basic details about its distribution over groups.2 "We can't share it" changes what you show, not whether you document it.
- Models change between versions. The paper's toxicity example shows two versions of one model behaving very differently, and stresses having a model card "that is updated with each new model release".2
The authors are frank about the limit: a model card's usefulness "relies on the integrity of the creator(s) of the card", and it is "one transparency tool among many", alongside third-party audits.2 A card is a claim; modules 400.8 and 400.9 teach how to make claims a stranger can check.
Worked example: a datasheet that blocks release
Big Widget's service team wants to fine-tune a model to summarise technicians' service notes. Its draft datasheet for the training set, bw-service-notes v1, reads:
| Part | Draft answer |
|---|---|
| Motivation | Created by the service analytics team to train a service-note summariser. Funded from the service budget. |
| Composition | 40,000 notes from 2024 to 2026, English only. Some notes include customer names. |
| Collection | (blank) |
| Consent | "Data is internal, so consent is not needed." |
| Licence | (blank) |
| Known gaps | No notes from plants abroad. |
Two blanks block release. Collection is empty: nobody can say how the notes were gathered, by whom or over what period, so nobody can check that the sample matches the job. Licence is empty, and the team plans to mix in a public instruction dataset from a dataset hub; until someone traces that dataset's licence (section 3), nobody knows what the team may do with the result. The consent answer is also weak: the notes are about customers and written by technicians, so they relate to people,1 and "internal" does not answer whether people were told or agreed. If nobody knows yet how the notes were collected, the honest entry is "not known yet", with the name of the person who will find out and by when. A blank is not an answer. "Not known yet" is, and it always comes with someone who will find out, but it still holds the release until it is filled.
The matching model card must then cover, at least: intended use (drafting summaries a technician checks), out-of-scope use (no warranty decisions), factors (English-only notes, no plants abroad), metrics with intervals, the evaluation data and the documents behind it, and the training data.2 This course adds two links the paper does not require: the card names its training data by datasheet version, and it names the base model or checkpoint the fine-tune started from. Section 6 shows why both links matter.
3. Trace the licence from source to model
A second common belief: "The licence label on a dataset hub is reliable." A 2023 audit says to check it.
Shayne Longpre and colleagues, a team of legal and machine-learning experts, audited and traced "1800+ text datasets", following each one's lineage "from their source, creators, series of license conditions, properties, and subsequent use".3 They found "frequent miscategorization of licenses on widely used dataset hosting sites, with license omission of 70%+ and error rates of 50%+".3 Their re-annotation found that 66% of the Hugging Face licences they analysed were in a different use category from the original author's licence, "often labeled as more permissive" than the author intended.3 (These figures are from the 2023 arXiv version, v3. The study was published in Nature Machine Intelligence in 2024; its figures are not checked against the published version here.)
Why the labels drift: popular collections are often treated as one block "instead of a lineage of data sources", re-packaged "(and re-licensing)" by one practitioner after another.3 Each hop can drop or change a condition.
How the audit traced a licence. The team collected every self-reported licence (from GitHub, arXiv, Hugging Face, Papers with Code and the collection itself), searched for a licence given to the data rather than to the accompanying code, and sorted each licence by three features: is use limited to academic or non-commercial purposes, must the source be attributed, and must derivatives carry the same terms. Where a dataset had several licences, each feature took "the strictest across licenses".3 They also note that their protocol "does not constitute legal advice", and that whether a licence can be enforced depends on several factors, including whether the dataset's creator holds rights in the data at all.3
A provenance chain for a model has one link per hop:
- Original source: where the text or images first came from, and its terms.
- Dataset: who assembled it, under what licence, and what the datasheet says.
- Collection or hub copy: the label where you downloaded it, and whether it matches link 2.
- Your training set: which version you used, what you added or removed.
- Your model: which training set version it was trained on, recorded in the model card.
Flag a chain when a link is missing (no licence found for the data, only for the code), or when two links contradict each other (the hub says one thing, the source another).
Worked example: the hub says yes, the source says no
The team's public instruction dataset is labelled on a hub as allowing commercial use. The team traces it: the dataset's own paper links a licence for the data that limits it to non-commercial use, and the dataset was built partly from another collection whose terms the team has not yet found.
- What the team may not do: treat the hub label as permission. The chain contradicts itself at link 3, and link 1 is missing.
- What the audit's method does and does not tell you: the audit sorted each licence feature by the strictest licence it found.3 That is a way of categorising datasets for research. It is not a ruling on which licence legally applies. Several licences on one dataset may be alternatives, may all apply at once, or may cover different parts of it; working out which is a legal question.
- What the team does: it borrows the caution, not the verdict. The gate answer for commercial training is hold: wait for a person. The team records the contradiction and the missing link in the datasheet and asks Big Widget's legal lead for a decision. The legal lead decides what the terms actually allow and records the decision and its basis. Until then the dataset stays out of any commercial training run.
That rule, hold on the strictest reading until a named person decides, is this course's rule. It is not legal advice, and it is not a finding of the audit.
4. One law as a dated example
Some laws now set data documentation duties for some systems. One example, with care about dates. Article 10 of the EU AI Act, as adopted in 2024 (Regulation (EU) 2024/1689), applies to "high-risk AI systems which make use of techniques involving the training of AI models with data".4 It says their training, validation and testing data sets "shall be subject to data governance and management practices" covering, among other things: "the relevant design choices"; "data collection processes and the origin of data, and in the case of personal data", "the original purpose of the data collection"; data-preparation operations "such as annotation, labelling, cleaning"; an examination for possible biases; and "the identification of relevant data gaps or shortcomings".4 The same article says such data sets "shall be relevant, sufficiently representative" and, "to the best extent possible, free of errors and complete in view of the intended purpose".4
Read the overlap with a datasheet: origin of data, original purpose, preparation, gaps. A team that writes datasheets is already collecting much of what this article asks about.
Dates and limits. The AI Act was amended by Regulation (EU) 2026/1744 of 8 July 2026. The amendment rewrites the Act's application dates (Article 113), naming 2 December 2027 "as regards AI systems classified as high-risk pursuant to Article 6(2) and Annex III" and 2 August 2028 for those under Article 6(1) and Annex I.4 Whether and how the 2026 amendment changes Article 10 itself is not checked in this module. Treat the 2024 text above as a historical example of what a data-governance rule can ask for, and check the amended text, and whether it applies to your system, before treating any of it as a current rule. As of November 2026.
5. How bad data reaches a model
Figure data as text
After bad data is found
Five steps in order. The order is the course’s procedure, built from its sources.
- HOLD — stop use and hold: freeze runs and releases; a named owner holds or constrains every model already in use. (course)
- NOTIFY — tell those affected: dataset maintainers, people policy and law name, and anyone given copies. [s5]
- REMOVE — clean every copy: including derived sets and caches; check new downloads match, for example by hash. [s4, s5]
- TRACE — retrain or retire: every model trained on it, and every model built on those. [s3, s4]
- RECORD — update the paperwork: datasheet erratum and new version; model card for every affected model. [s1, s2]
Objective: 400.3-c. Text equivalent of figures/after-bad-data-is-found.svg.
A third common belief: "Bad items in a huge dataset are too rare to matter." Two pieces of research show why that is wrong in two different ways.
Unlawful content. In 2023 David Thiel, with Jeffrey Hancock, examined the LAION-5B dataset, "parts of which were used to train the popular Stable Diffusion series of models".5 Using "PhotoDNA perceptual hash matching, cryptographic hash matching, k-nearest neighbors queries and ML classifiers", the study detected many hundreds of instances of known child sexual abuse material (CSAM) in the training set, plus new candidates that outside parties later verified.5 This course teaches only the data-governance lesson; it never describes or shows the material. The lesson: in a dataset collected from the web at this scale, nobody had checked every item, and unlawful items were in the data that models were trained on. The study's recommendations cover maintaining copies of the training set, building future training sets, altering existing models, and the hosting of models trained on LAION-5B.5
Be exact about what this shows. Only parts of LAION-5B were used to train Stable Diffusion, and the study describes its aim as an attempt "to measure to what degree CSAM itself may have played a role in the training process".5 The abstract does not say that every detected item was in every model's training data, or that any model memorised or can reproduce it. What it does show is that the problem does not stay in one file. Every copy of the dataset, and every model trained on it, is exposed, and has to be traced and assessed rather than assumed clean.
Poisoned content. Nicholas Carlini and colleagues showed in 2023 that an attacker can deliberately plant data in web-scale datasets.6 At the scale of billions of samples crawled from the internet, it is "infeasible to manually curate" each example.6 Many such datasets are published not as fixed files but as an index of web addresses that each user downloads for themselves, or as periodic snapshots of web content.6 That opens two attacks:
- Split-view poisoning exploits "the mutable nature of internet content": what the dataset's curator saw at collection time can differ from what a later user downloads. One route is to buy expired web domains that the dataset still points to, and serve different content from them.6 The authors showed they "could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just $60 USD" in 2023.6
- Frontrunning poisoning targets datasets that "periodically snapshot crowd-sourced content", such as Wikipedia: an attacker times an edit just before the snapshot, and even if moderators revert it later, it stays in the snapshot used for training.6
Why 0.01% matters: the authors note that earlier research found that "even minuscule amounts of adversarial noise" in training sets can introduce targeted mistakes into a model.6 They also report that, to their knowledge, no real-world poisoning of web-scale datasets had yet occurred: this is a demonstration of what was possible, not a record of an attack.6
So "too rare to matter" fails twice. Unlawful content is not made acceptable by being rare; that is this course's framing. It can reach models trained on the data, which is why the study's recommendations extend to existing models and their hosting.5 Poisoned content is placed on purpose, and a tiny, cheap fraction can be enough.6
6. What removal requires
Deleting the bad rows from one file is not removal. The data has copies, and models have already learned from it.
Why a model is the hard part. The provenance audit notes that training models on data is "both expensive and largely irreversible".3 So this course plans for two fixes once a model has been trained on bad data: retrain it, or retire it. Retraining means starting again from a cleaned dataset and from a starting checkpoint that never saw the bad data. Fine-tuning the affected model again on clean data is not a retrain. (Methods that try to make a model "forget" specific data are outside this module and not checked here.)
Defences before the fact. Against split-view poisoning, the authors propose integrity verification: "distributing cryptographic hashes for all content", so users can check they received what the curator indexed. Against frontrunning, they propose randomising the order of snapshots or delaying content before it enters a snapshot.6 They note a limit: integrity checks also reject harmless changes such as re-encoding, resizing or cropping images.6 After their disclosure, six of the ten datasets they studied followed their recommended integrity checks, and they notified Wikipedia about the frontrunning risk.6
Read a hash check for what it is. A mismatch means the bytes you received differ from the bytes the curator recorded. It does not tell you why: the cause could be an attack or a harmless re-encoding.6 Reject or quarantine the item, then find out. A match means you received what was recorded. It does not mean the recorded item was lawful, accurate or safe. A hash makes a dataset tamper-evident, not tamper-proof, and it says nothing about whether the original was acceptable.
The steps after a dataset is found to contain unlawful or poisoned material. This order is this course's procedure, built from the sources above; it is not a rule from any one of them, and it is not legal advice:
- Stop use and hold. Freeze training runs and releases that use the dataset or any copy. Freezing releases does not stop a model that is already serving, so also deal with every exposed model already in use: in production, hosted elsewhere, or shared with a supplier. A named owner decides at once, for each one, whether it is held (taken out of service) or constrained (kept running only within written limits) while it is assessed, and records the decision and the reason. This course's default is hold. Only a named person can lift a hold, and the record says when they will look at it again. A hold nobody returns to decays to deny; it never drifts into allow.
- Notify. Tell the dataset's maintainers, the people your organisation's policy and the law name, and anyone you shared copies or models with, such as a supplier. Carlini and colleagues, for example, notified the maintainers of each affected dataset.6
- Remove. Remove the items from every copy you hold, including derived training sets and caches, and check that new downloads match the cleaned version, for example by hash.56
- Trace, then retrain or retire. Find every model trained on an affected version, and every model built on top of one of those, such as a fine-tune or an adapter. Retrain each from the cleaned data and a clean starting point, or retire it, including any hosted and supplier copies.35
- Record. Update the datasheet (an erratum, a new version, and how the old version is retired) and the model card for every affected model.12
Keep an incident log from step 1 onward. "Record" is last in the list because it closes the incident, not because you wait until the end to write things down.
At Big Widget, the paper trail makes step 4 possible. If every model card names the training-set version it used and the model it started from, the team can follow both kinds of link and list the exposed models. If a card names only its training data, a fine-tune built on an affected base model but trained on clean data is missed. Any model whose lineage is not known yet goes on the list as "not known yet" and stays on hold until someone finds out.
7. At Big Widget Company: the three yeses for data
These are the data parts of the three yeses from module 100.9: Get to Yes · Stay at Yes · Recover to Yes. A complete datasheet is evidence, not permission. A named person says yes; a model, a documentation tool or a passing check does not. Every gate answer is one of four: allow, constrain, hold or deny (go ahead, only this far, wait for a person, no).
- Get to Yes (before training). Every dataset has a datasheet with no blank collection or licence answer, checked by someone other than its writer. Every licence chain is traced to its source. "Not known yet" is allowed, with an owner and a date, but it holds the release. A contradiction holds the dataset until the legal lead decides. The service lead owns the release of the model.
- Stay at Yes (while it runs). When a dataset changes, its datasheet and version change. Every model card names the dataset versions and the starting model it used. The first yes is not the last check.
- Recover to Yes (when something is found). Stop and hold, notify, remove, trace and retrain or retire, record. Holding a model stops the exposure; it is not, by itself, a return to yes. Call it recovered only when the records show every exposed model, including hosted and supplier copies, and what was done to each, and the named owner agrees that list is complete.
What to keep, and what to share. Keep datasheets, licence traces, model cards and the incident log in the team's records, versioned and access-controlled. A datasheet describes personal data; it does not copy it. Do not paste customer names or the content of service notes into it. A receipt of a gate decision holds no private content: the decision, who made it, when, and references to the evidence, such as dataset versions, manifest hashes and model-card versions. Not the notes, not customer names, and never the affected items or links to them. A reference in a receipt points at the controlled record; it does not give anyone the right to see that record. Here the receipt can be an unsigned decision record. Modules 400.8 and 400.9 show how to make a signed receipt something a stranger can check, and what it does and does not prove.
My checklist:
- I can show a datasheet for every dataset this model used, with no blank boxes and an owner for every "not known yet".
- I have traced each licence to its source, and a named person decided every contradiction.
- I know which dataset versions and which starting model each model card names.
- I know who can hold a model that is already running, and when each hold is reviewed.
- My receipts hold no private content.
Words to know: datasheet, model card, provenance, lineage, licence chain, split-view poisoning, frontrunning poisoning, integrity hash, erratum, hold, receipt.
Further learning (optional, not sources)
These free university materials go deeper. They are pointers only; nothing in this module is copied from them, and none is a source for any claim here.
- Stanford University, CS324: Large Language Models (Winter 2022), lectures on data and legality.
- University of Cambridge, Advanced Data Science, Practical 1: access and assess, data-pipeline background for tracing where data came from (not licensing or consent).