
Data with a paper trail · Technical gear
Reviewed and published, version 1.0.0. Facts as of 18 Nov 2026; wording revised 19 Nov 2026. Not a credential. Research results are dated and hold for the settings their papers tested. The legal text is a dated, historical example; nothing here is legal advice. Where we did not check something, we say "not checked".
You will be able to:
- write a datasheet for a training or evaluation dataset (motivation, composition, collection, consent, licence, known gaps) and a model card for the model trained on it, linked by version;
- trace a dataset's provenance and licence as a graph from source to model, and flag missing and contradictory links;
- explain the mechanisms by which unlawful or poisoned content in a web-scale dataset reaches trained models (index-based distribution, split-view and frontrunning poisoning), and what removal requires.
Big Widget Company and its data are made up for teaching.
1. The datasheet as a versioned artifact
Gebru et al. propose that every dataset be accompanied by a datasheet that "documents its motivation, composition, collection process, recommended uses, and so on", by analogy with electronic component datasheets.1 The question set is organised by lifecycle stage: motivation, composition, collection process, preprocessing/cleaning/labelling, uses, distribution and maintenance.1 The audience is two stakeholder groups, dataset creators and dataset consumers.1
The fields this module requires, mapped to the paper's questions:1
| Field | Paper section | Minimum content |
|---|---|---|
motivation | Motivation | Purpose; creator and the entity it acted for; funder |
composition | Composition | Instance type and counts; sample vs. population and representativeness; labels; missing information; recommended splits; errors, noise, redundancy; external resources and their guarantees and restrictions; confidential and sensitive data; subpopulations; re-identifiability |
collection | Collection process | Acquisition (observed, reported, inferred), mechanisms and how they were validated, sampling strategy, who collected and how they were paid, timeframe, ethical review |
consent | Collection process (data about people) | Collected directly or via third parties; notice given; consent obtained, with the exact language; revocation mechanism; impact analysis |
licence | Distribution | Licence or terms of use; third-party IP or other restrictions; export or regulatory restrictions |
known_gaps | Composition; Uses; Maintenance | Missing information; composition or collection issues affecting future use; tasks the data should not be used for; erratum |
Rules the paper sets that change engineering practice:
- Scope of "relates to people". "Any dataset containing text that was written by people relates to people", so the consent questions apply to logs, tickets and notes.1
- Unknown is a value; null is not. The paper's example datasheet answers some questions "Unknown to the authors of the datasheet", and the authors recommend answering as many questions as possible rather than skipping the datasheet.1 Encode
unknownexplicitly and let a release check fail on empty required fields, not onunknown. - Not auto-generated. The process "is not intended to be automated".1 Tooling can pre-fill counts and hashes; the answers to why, how and under what terms need a person.
- Maintenance. The maintenance questions ask about errata, updates (including deleted instances) and how obsolescence of older versions is communicated.1 The authors recommend updated datasheets for datasets that change only infrequently, and flag that their questions and workflow "may pose problems for dynamic datasets".1 The paper does not solve documentation for continuously changing data.
Operational rules that are this course's design, not the paper's:
- One datasheet version per dataset version. Every dataset version bump produces a datasheet version bump, however frequent. If the data changes too fast for a person to re-answer the questions each time, record that limit in
known_gapsand name the owner who decides the documentation cadence; do not let tooling silently regenerate the human-answered fields. - Manifest linkage. Store the datasheet next to a manifest of the dataset version, a list of
(item id, content hash)pairs, and record the manifest's own hash in the datasheet. Section 6 shows what content hashes do and do not prove. - Writer ≠ checker. The release check on a datasheet is approved by someone other than its writer.
2. The model card, linked to the datasheet
Mitchell et al. propose model cards: short (one to two page) records that accompany a trained model and are "complements" to datasheets.2 Their sections are model details (including version and licence), intended use (with out-of-scope uses), factors, metrics (performance measures, decision thresholds, approaches to uncertainty and variability), evaluation data, training data, quantitative analyses (unitary and intersectional), ethical considerations, and caveats and recommendations.2
Precision points for an engineer:
- Uncertainty is part of the metric. The metrics section asks how measurements are calculated, for example with confidence intervals, and quantitative analyses should give confidence interval values when possible.2 The paper's example card reports 95% confidence intervals from bootstrap resampling.2
- Disaggregate. Quantitative analyses should be broken down by the chosen factors, alone and in intersection.2
- Data references. Evaluation datasets should ideally point to documents giving "visibility into the source and composition of the dataset".2 If training data cannot be disclosed (proprietary, under NDA), the card still gives basic details about its distribution over groups.2
- Per release. The paper's toxicity example shows two versions of one classifier and argues for a card "updated with each new model release".2
- Limits. The card's usefulness "relies on the integrity of the creator(s)", and it is "one transparency tool among many", alongside third-party algorithmic audits.2
For Big Widget's service-note summariser, and as this course's design rather than the paper's, the card's evaluation-data and training-data sections name datasets by name@version and manifest hash, for example bw-service-notes@1.2 (manifest sha256:…), and its model-details section names the starting model by name@version (base model, checkpoint or adapter parent). Both fields are needed: the lineage query in section 6 follows data edges and model-to-model edges.
Release check on a draft datasheet
bw-service-notes v1 has collection: null and licence: null, and consent: "internal, not needed". A release check fails it on the two nulls: without the collection process, nobody can judge representativeness or sampling; without the licence, the right to train on and ship the data is unknown. Replacing a null with unknown plus an owner and due date makes the record honest, but the gate answer is still hold until a person fills it. The consent value is also non-compliant with the paper's scope rule, because technician-written notes about customers relate to people.1 The model card cannot be completed either: its training-data section would point at a datasheet with no collection or licence entries.2
3. Provenance as a graph
Longpre et al. (legal and ML experts) audited and traced "1800+ text datasets", recording lineage "from their source, creators, series of license conditions, properties, and subsequent use"; the audit covered 44 fine-tuning data collections comprising 1,858 datasets.3 Findings on licence metadata:
- "frequent miscategorization of licenses on widely used dataset hosting sites, with license omission of 70%+ and error rates of 50%+";3
- 70%+ of licences for popular datasets on GitHub and Hugging Face were "Unspecified";3
- on re-annotation, 66% of analysed Hugging Face licences were in a different use category from the original author's licence, "often labeled as more permissive".3
These figures are from arXiv v3 (4 Nov 2023). The study was published in Nature Machine Intelligence in 2024; the published figures are not checked against v3 here.
Model it as a directed acyclic graph. Nodes are original sources, datasets, collections or hub copies, training-set versions and models. An edge u → v means v was derived from u. That includes model-to-model edges: base model → fine-tune, checkpoint → continued training, base → adapter, teacher → distilled student. Each dataset node carries the terms asserted at that node. The audit stresses tracing each stage of the lifecycle, collecting information "at each stage, not just the last", because collections are often treated as monolithic rather than as "a lineage of data sources" re-packaged "(and re-licensing)" at each hop.3
Licence categories. The audit sorts licences by three features relevant to the model lifecycle: permitted use (is use limited to academic or non-commercial purposes?), attribution, and share-alike. Where a dataset has several licences, each feature is set to "the strictest across licenses".3 Order each feature by restrictiveness (for permitted use: commercial < non-commercial or academic only) and take the most restrictive value. Applying the same maximum along every path from source to training set, and across every dataset in a training mixture, is this course's conservative extension of that rule, not a finding of the paper.
Be clear what the maximum is. In the audit it is a categorisation rule. It is not a determination of which licence legally governs. Multiple licences can be alternatives (the user may pick one), cumulative (all apply) or scoped to different components, and a plain max ignores those distinctions. The course uses it as a hold rule: the strictest reading stands until a named legal decision resolves the actual permissions and records its basis on the edge.
Flags.
- Missing: a node on the path has no terms for the data. The audit counts a licence as found only when it is given to the data itself (not just the accompanying code) by its authors.3
- Contradiction: two nodes assert terms in different categories for the same data, for example a hub label of commercial-allowed downstream of an author licence of non-commercial.
- Not legal advice: the audit itself says its protocol "does not constitute legal advice", and that enforceability depends on factors including whether the dataset creator holds a copyright interest in the data at all.3 A contradiction is resolved by a named person in legal, and the resolution is recorded on the edge.
Worked trace. Big Widget's mixture adds a public instruction dataset whose hub label reads commercial-allowed. The dataset's paper links a data licence restricting it to non-commercial use, and one upstream source has no terms found. Path result: permitted use = non-commercial (strictest wins); one missing flag; one contradiction flag. Gate answer under the course rule: hold for any commercial training run until Big Widget's legal lead records a decision and its basis; both flags written into the datasheet's licence and known_gaps fields.
4. A dated legal example: Article 10 of the 2024 AI Act
Regulation (EU) 2024/1689, Article 10, in its original 2024 text, applies to "high-risk AI systems which make use of techniques involving the training of AI models with data".4 Article 10(2) requires data governance and management practices; mapped to datasheet fields:4
| Art. 10(2), 2024 text | Datasheet field |
|---|---|
| (a) "the relevant design choices" | motivation |
| (b) "data collection processes and the origin of data, and in the case of personal data", "the original purpose of the data collection" | collection, consent, provenance graph |
| (c) data-preparation operations "such as annotation, labelling, cleaning" | preprocessing |
| (d) "the formulation of assumptions" about what the data measure and represent | motivation, composition |
| (e) "an assessment of the availability, quantity and suitability of the data sets" | composition |
| (f), (g) examination for possible biases, and measures "to detect, prevent and mitigate" them | composition, known_gaps; module 400.7 |
| (h) "the identification of relevant data gaps or shortcomings" | known_gaps |
Article 10(3) adds that the data sets "shall be relevant, sufficiently representative" and, "to the best extent possible, free of errors and complete in view of the intended purpose".4
Dates. Regulation (EU) 2026/1744 of 8 July 2026 amends Regulation (EU) 2024/1689. Its rewrite of Article 113 names 2 December 2027 "as regards AI systems classified as high-risk pursuant to Article 6(2) and Annex III" and 2 August 2028 for those under Article 6(1) and Annex I.4 Whether and how the amendment changes Article 10 is not checked in this module. Use the 2024 text only as a historical example; check the amended text and its applicability to a given system before stating a current obligation. As of November 2026.
5. How unlawful and poisoned content reaches a model
Figure data as text
After bad data is found
Five steps in order. The order is the course’s procedure, built from its sources.
- HOLD — stop use and hold: freeze runs and releases; a named owner holds or constrains every model already in use. (course)
- NOTIFY — tell those affected: dataset maintainers, people policy and law name, and anyone given copies. [s5]
- REMOVE — clean every copy: including derived sets and caches; check new downloads match, for example by hash. [s4, s5]
- TRACE — retrain or retire: every model trained on it, and every model built on those. [s3, s4]
- RECORD — update the paperwork: datasheet erratum and new version; model card for every affected model. [s1, s2]
Objective: 400.3-c. Text equivalent of figures/after-bad-data-is-found.svg.
Scale removes inspection. Web-scale datasets hold billions of crawled samples, and at that scale "it is infeasible to manually curate and ensure the quality of each example".5
Unlawful content: LAION-5B (2023). David Thiel (author), with Jeffrey Hancock as research team head, examined LAION-5B, "parts of which were used to train the popular Stable Diffusion series of models", combining "PhotoDNA perceptual hash matching, cryptographic hash matching, k-nearest neighbors queries and ML classifiers". They detected many hundreds of instances of known CSAM in the training set, plus new candidates later verified by outside parties, and made recommendations covering maintenance of copies of the training set, construction of future training sets, alteration of existing models, and hosting of models trained on LAION-5B.6 This course teaches the data-governance lesson only. The engineering point is the exposure path: dataset → copies → training sets → models → derived models → hosted copies. Each node downstream is exposed and must be traced and assessed. Exposure is not verified inclusion: only "parts of" LAION-5B trained Stable Diffusion, and the study describes an attempt "to measure to what degree" the material "may have played a role in the training process".6 The abstract does not establish that each detected item was in a given training subset, or was memorised or reproduced by a given model. Section 6's response treats exposure conservatively as a matter of course policy, not as a measured result.
Poisoning: two attacks (Carlini et al., 2023). Many web-scale datasets are distributed not as static artifacts but as an index of web content each client crawls, or as periodic snapshots.5 Let the index be {(url_i, caption_i)} for i = 1..N, with content x_i observed by the curator at time t_0, and x_i' fetched by a client at time t_1.
- Split-view poisoning. Integrity requires
x_i' = x_i. Without published cryptographic integrity data, nothing guarantees it, and the attacker controlsx_i'for everyurl_ion a domain they control.5 The practical route is buying expired domains still referenced by the index; the authors rank domains by images hosted per unit cost.5 The poisoned fraction isp = |{i : domain(url_i) ∈ B}| / Nfor the bought setB. They report that for USD 60 they "could have poisoned 0.01% of the LAION-400M or COYO-700M datasets" in 2023.5 - Frontrunning poisoning. For snapshot-based datasets, if the snapshot time
t_iof a page is predictable, an edit made shortly beforet_ipersists in the snapshot even when moderators later revert it.5 For Wikipedia, the authors report the snapshot time of an article can be predicted "down to the minute".5
Why small p matters: the authors cite prior work showing that "even minuscule amounts of adversarial noise" in training sets suffice to introduce targeted mistakes.5 They also state that, to their knowledge, no real-world poisoning of web-scale datasets had occurred: the result is a feasibility demonstration under stated assumptions, not an incident record.5
What existing hashes did and did not protect (the paper's survey of ten datasets):5
- LAION-5B's released hashes covered the text of the URL and caption, "not the contents of the URL", so they do not protect the image.
- COYO-700M images came with a perceptual hash (pHash) meant to tolerate benign image changes, which the authors say is not adversarially robust.
- FaceScrub and PubFig did ship cryptographic hashes, but without an official downloader that verifies them, and nearly all third-party downloaders ignored them.
6. What removal requires
Defences before the fact.5
- Integrity verification: distribute a cryptographic hash
h_i = H(x_i)for every item, so a client can rejectx_i'withH(x_i') ≠ h_i. Limitation: it also rejects benign re-encoding, resizing or cropping. After disclosure, six of the ten datasets studied adopted the recommended integrity checks, and the authors provided patches to a widely used downloader. - Timing defences against frontrunning: randomise snapshot order, or delay content and apply trusted moderators' reversions before it enters a snapshot.
What a hash check proves. H(x_i') ≠ h_i proves only that the received bytes differ from the recorded bytes. It does not distinguish an adversarial swap from a benign re-encode, resize or crop, which the authors name as the limitation above.5 Treat a mismatch as reject or quarantine, then investigate, not as proof of attack. H(x_i') = h_i proves integrity relative to what the curator recorded at t_0; it says nothing about whether x_i itself was lawful, accurate or safe. Integrity is tamper-evidence, not content safety.
After the fact, the model is the expensive node. The provenance audit describes training on data as "both expensive and largely irreversible".3 The fixes this course plans for are retraining or retiring. A retrain starts from a cleaned dataset version and a starting checkpoint outside the exposed set A defined below; continuing to train an exposed checkpoint on clean data is not a retrain. Approximate "unlearning" methods are out of scope and not checked here.
Response procedure. The order is this course's procedure, assembled from the sources; it is not a rule from any one of them and not legal advice. Let D_bad = {d@v : v ∈ V_bad} and let A = Desc(D_bad) ∩ Models: every model reachable from an affected dataset version through data edges or model-to-model edges (section 3).
- Stop use and hold. Block training jobs and releases that read any node in
D_bador its copies. Blocking jobs does not stop a model already serving, since inference reads no dataset. So for everym ∈ Athat is deployed, hosted or held by a supplier, a named owner records an interim decision at once: hold (out of service) or constrain (running within written limits), with the reason and a review date. The course default is hold; lifting a hold needs a named person, and a hold past its review date with no decision decays to deny, never to allow. Models whose lineage is unknown are treated as inAuntil shown otherwise. - Notify. The maintainers, whoever policy and law require, and every downstream recipient of a copy or model (for example a supplier). Carlini et al. notified the maintainers of each affected dataset and notified Wikipedia.5
- Remove. Publish
d@v'with the items removed, a new manifest and hashes; purge every copy and derived training set; verify downloads againstv'hashes.65 - Retrain or retire. For each
m ∈ A, including hosted and supplier copies, retrain fromv'and a clean starting point not inA, or retire it.36 - Record. Datasheet erratum, new version, and how the old version's obsolescence is communicated; a new model card version for every retrained model.12
Log each step as it happens; step 5 closes the incident, it is not the first write.
Completeness. Computing A is only a query if section 2's linkage exists for both edge types. The counterexample to a data-only query: d@1 → B (base model trained on bad data), then c@1, B → F (fine-tune on a clean dataset). F's card names only c@1, which does not descend from d@1, so a query over training-data fields misses F, though its weights descend from B. With the starting-model field, F ∈ A. Without name@version, manifest hashes and starting-model links in the cards, A is unknown, and removal cannot be shown to be complete.
7. Three yeses, as checks
The data parts of the three yeses (module 100.9): Get to Yes · Stay at Yes · Recover to Yes. A passing check is evidence, not permission; a named person gives each gate answer (allow, constrain, hold or deny). No model, CI job or documentation generator releases a dataset or a model: the checks below can only fail a release or route it to that person.
- Get to Yes: release holds if any required datasheet field is null or
unknown, if the datasheet was approved by its own writer, or if the licence graph has an unresolved missing or contradiction flag on any path into the training set. The legal lead resolves licence flags; the service lead owns the model release. - Stay at Yes: every dataset change bumps its version, manifest hash and datasheet; every model card names its datasets by version and hash and its starting model by version.
- Recover to Yes: the five-step procedure, closed only when every
m ∈ Ahas a recorded outcome (retrained or retired), every interim hold is accounted for, and the named owner agrees the setAis complete. A hold stops exposure; it is not itself a return to yes.
Receipts. A receipt of a gate decision holds no private content. Example fields: decision, decided_by, time, dataset (name@version), manifest_sha256, model_card (name@version). Never item contents, customer names, service-note text, or the affected items, their URLs or their hashes. The incident log and datasheets live in access-controlled records, not in the receipt. A reference or hash in a receipt is not a licence to disclose what it points at, and a hash of a small or guessable record should not be assumed to be anonymous. At this module's stage the receipt can be an unsigned decision record; modules 400.8 and 400.9 cover signing and making receipts verifiable by a stranger offline, and what such a receipt does not prove.
Terms: datasheet, model card, provenance graph, lineage, licence category (permitted use, attribution, share-alike), split-view poisoning, frontrunning poisoning, cryptographic vs. perceptual hash, manifest, erratum, exposed set, hold, receipt.
Further learning (optional, not sources)
Pointers only; nothing here is copied from them and none supports a claim in this module.
- Stanford University, CS324: Large Language Models (Winter 2022), lectures on data and legality.
- University of Cambridge, Advanced Data Science, Practical 1: access and assess, data-pipeline background (not licensing or consent).