Level 400: Engineering AI systems · Module 400.3

Two colleagues trace a page in an open binder beside plain file boxes on a café table.
Show me the paper trail

Data with a paper trail · Technical gear

Reviewed and published, version 1.0.0. Facts as of 18 Nov 2026; wording revised 19 Nov 2026. Not a credential. Research results are dated and hold for the settings their papers tested. The legal text is a dated, historical example; nothing here is legal advice. Where we did not check something, we say "not checked".

You will be able to:

Big Widget Company and its data are made up for teaching.

1. The datasheet as a versioned artifact

Gebru et al. propose that every dataset be accompanied by a datasheet that "documents its motivation, composition, collection process, recommended uses, and so on", by analogy with electronic component datasheets.1 The question set is organised by lifecycle stage: motivation, composition, collection process, preprocessing/cleaning/labelling, uses, distribution and maintenance.1 The audience is two stakeholder groups, dataset creators and dataset consumers.1

The fields this module requires, mapped to the paper's questions:1

FieldPaper sectionMinimum content
motivationMotivationPurpose; creator and the entity it acted for; funder
compositionCompositionInstance type and counts; sample vs. population and representativeness; labels; missing information; recommended splits; errors, noise, redundancy; external resources and their guarantees and restrictions; confidential and sensitive data; subpopulations; re-identifiability
collectionCollection processAcquisition (observed, reported, inferred), mechanisms and how they were validated, sampling strategy, who collected and how they were paid, timeframe, ethical review
consentCollection process (data about people)Collected directly or via third parties; notice given; consent obtained, with the exact language; revocation mechanism; impact analysis
licenceDistributionLicence or terms of use; third-party IP or other restrictions; export or regulatory restrictions
known_gapsComposition; Uses; MaintenanceMissing information; composition or collection issues affecting future use; tasks the data should not be used for; erratum

Rules the paper sets that change engineering practice:

Operational rules that are this course's design, not the paper's:

2. The model card, linked to the datasheet

Mitchell et al. propose model cards: short (one to two page) records that accompany a trained model and are "complements" to datasheets.2 Their sections are model details (including version and licence), intended use (with out-of-scope uses), factors, metrics (performance measures, decision thresholds, approaches to uncertainty and variability), evaluation data, training data, quantitative analyses (unitary and intersectional), ethical considerations, and caveats and recommendations.2

Precision points for an engineer:

For Big Widget's service-note summariser, and as this course's design rather than the paper's, the card's evaluation-data and training-data sections name datasets by name@version and manifest hash, for example bw-service-notes@1.2 (manifest sha256:…), and its model-details section names the starting model by name@version (base model, checkpoint or adapter parent). Both fields are needed: the lineage query in section 6 follows data edges and model-to-model edges.

Release check on a draft datasheet

bw-service-notes v1 has collection: null and licence: null, and consent: "internal, not needed". A release check fails it on the two nulls: without the collection process, nobody can judge representativeness or sampling; without the licence, the right to train on and ship the data is unknown. Replacing a null with unknown plus an owner and due date makes the record honest, but the gate answer is still hold until a person fills it. The consent value is also non-compliant with the paper's scope rule, because technician-written notes about customers relate to people.1 The model card cannot be completed either: its training-data section would point at a datasheet with no collection or licence entries.2

3. Provenance as a graph

Longpre et al. (legal and ML experts) audited and traced "1800+ text datasets", recording lineage "from their source, creators, series of license conditions, properties, and subsequent use"; the audit covered 44 fine-tuning data collections comprising 1,858 datasets.3 Findings on licence metadata:

These figures are from arXiv v3 (4 Nov 2023). The study was published in Nature Machine Intelligence in 2024; the published figures are not checked against v3 here.

Model it as a directed acyclic graph. Nodes are original sources, datasets, collections or hub copies, training-set versions and models. An edge u → v means v was derived from u. That includes model-to-model edges: base model → fine-tune, checkpoint → continued training, base → adapter, teacher → distilled student. Each dataset node carries the terms asserted at that node. The audit stresses tracing each stage of the lifecycle, collecting information "at each stage, not just the last", because collections are often treated as monolithic rather than as "a lineage of data sources" re-packaged "(and re-licensing)" at each hop.3

Licence categories. The audit sorts licences by three features relevant to the model lifecycle: permitted use (is use limited to academic or non-commercial purposes?), attribution, and share-alike. Where a dataset has several licences, each feature is set to "the strictest across licenses".3 Order each feature by restrictiveness (for permitted use: commercial < non-commercial or academic only) and take the most restrictive value. Applying the same maximum along every path from source to training set, and across every dataset in a training mixture, is this course's conservative extension of that rule, not a finding of the paper.

Be clear what the maximum is. In the audit it is a categorisation rule. It is not a determination of which licence legally governs. Multiple licences can be alternatives (the user may pick one), cumulative (all apply) or scoped to different components, and a plain max ignores those distinctions. The course uses it as a hold rule: the strictest reading stands until a named legal decision resolves the actual permissions and records its basis on the edge.

Flags.

Worked trace. Big Widget's mixture adds a public instruction dataset whose hub label reads commercial-allowed. The dataset's paper links a data licence restricting it to non-commercial use, and one upstream source has no terms found. Path result: permitted use = non-commercial (strictest wins); one missing flag; one contradiction flag. Gate answer under the course rule: hold for any commercial training run until Big Widget's legal lead records a decision and its basis; both flags written into the datasheet's licence and known_gaps fields.

4. A dated legal example: Article 10 of the 2024 AI Act

Regulation (EU) 2024/1689, Article 10, in its original 2024 text, applies to "high-risk AI systems which make use of techniques involving the training of AI models with data".4 Article 10(2) requires data governance and management practices; mapped to datasheet fields:4

Art. 10(2), 2024 textDatasheet field
(a) "the relevant design choices"motivation
(b) "data collection processes and the origin of data, and in the case of personal data", "the original purpose of the data collection"collection, consent, provenance graph
(c) data-preparation operations "such as annotation, labelling, cleaning"preprocessing
(d) "the formulation of assumptions" about what the data measure and representmotivation, composition
(e) "an assessment of the availability, quantity and suitability of the data sets"composition
(f), (g) examination for possible biases, and measures "to detect, prevent and mitigate" themcomposition, known_gaps; module 400.7
(h) "the identification of relevant data gaps or shortcomings"known_gaps

Article 10(3) adds that the data sets "shall be relevant, sufficiently representative" and, "to the best extent possible, free of errors and complete in view of the intended purpose".4

Dates. Regulation (EU) 2026/1744 of 8 July 2026 amends Regulation (EU) 2024/1689. Its rewrite of Article 113 names 2 December 2027 "as regards AI systems classified as high-risk pursuant to Article 6(2) and Annex III" and 2 August 2028 for those under Article 6(1) and Annex I.4 Whether and how the amendment changes Article 10 is not checked in this module. Use the 2024 text only as a historical example; check the amended text and its applicability to a given system before stating a current obligation. As of November 2026.

5. How unlawful and poisoned content reaches a model

Five steps after a dataset is found to contain unlawful or poisoned material: stop use and hold, notify, remove from every copy, trace every affected model then retrain or retire it, and record the change in the datasheet and model cards.
The order is the course's procedure, built from its sources. Notify, as Carlini and colleagues did (s5); remove and check new downloads by hash (s4, s5); trace, then retrain or retire (s3, s4); update the datasheet and model cards (s1, s2).
Figure data as text

After bad data is found

Five steps in order. The order is the course’s procedure, built from its sources.

  1. HOLD — stop use and hold: freeze runs and releases; a named owner holds or constrains every model already in use. (course)
  2. NOTIFY — tell those affected: dataset maintainers, people policy and law name, and anyone given copies. [s5]
  3. REMOVE — clean every copy: including derived sets and caches; check new downloads match, for example by hash. [s4, s5]
  4. TRACE — retrain or retire: every model trained on it, and every model built on those. [s3, s4]
  5. RECORD — update the paperwork: datasheet erratum and new version; model card for every affected model. [s1, s2]

Objective: 400.3-c. Text equivalent of figures/after-bad-data-is-found.svg.

Scale removes inspection. Web-scale datasets hold billions of crawled samples, and at that scale "it is infeasible to manually curate and ensure the quality of each example".5

Unlawful content: LAION-5B (2023). David Thiel (author), with Jeffrey Hancock as research team head, examined LAION-5B, "parts of which were used to train the popular Stable Diffusion series of models", combining "PhotoDNA perceptual hash matching, cryptographic hash matching, k-nearest neighbors queries and ML classifiers". They detected many hundreds of instances of known CSAM in the training set, plus new candidates later verified by outside parties, and made recommendations covering maintenance of copies of the training set, construction of future training sets, alteration of existing models, and hosting of models trained on LAION-5B.6 This course teaches the data-governance lesson only. The engineering point is the exposure path: dataset → copies → training sets → models → derived models → hosted copies. Each node downstream is exposed and must be traced and assessed. Exposure is not verified inclusion: only "parts of" LAION-5B trained Stable Diffusion, and the study describes an attempt "to measure to what degree" the material "may have played a role in the training process".6 The abstract does not establish that each detected item was in a given training subset, or was memorised or reproduced by a given model. Section 6's response treats exposure conservatively as a matter of course policy, not as a measured result.

Poisoning: two attacks (Carlini et al., 2023). Many web-scale datasets are distributed not as static artifacts but as an index of web content each client crawls, or as periodic snapshots.5 Let the index be {(url_i, caption_i)} for i = 1..N, with content x_i observed by the curator at time t_0, and x_i' fetched by a client at time t_1.

Why small p matters: the authors cite prior work showing that "even minuscule amounts of adversarial noise" in training sets suffice to introduce targeted mistakes.5 They also state that, to their knowledge, no real-world poisoning of web-scale datasets had occurred: the result is a feasibility demonstration under stated assumptions, not an incident record.5

What existing hashes did and did not protect (the paper's survey of ten datasets):5

6. What removal requires

Defences before the fact.5

What a hash check proves. H(x_i') ≠ h_i proves only that the received bytes differ from the recorded bytes. It does not distinguish an adversarial swap from a benign re-encode, resize or crop, which the authors name as the limitation above.5 Treat a mismatch as reject or quarantine, then investigate, not as proof of attack. H(x_i') = h_i proves integrity relative to what the curator recorded at t_0; it says nothing about whether x_i itself was lawful, accurate or safe. Integrity is tamper-evidence, not content safety.

After the fact, the model is the expensive node. The provenance audit describes training on data as "both expensive and largely irreversible".3 The fixes this course plans for are retraining or retiring. A retrain starts from a cleaned dataset version and a starting checkpoint outside the exposed set A defined below; continuing to train an exposed checkpoint on clean data is not a retrain. Approximate "unlearning" methods are out of scope and not checked here.

Response procedure. The order is this course's procedure, assembled from the sources; it is not a rule from any one of them and not legal advice. Let D_bad = {d@v : v ∈ V_bad} and let A = Desc(D_bad) ∩ Models: every model reachable from an affected dataset version through data edges or model-to-model edges (section 3).

  1. Stop use and hold. Block training jobs and releases that read any node in D_bad or its copies. Blocking jobs does not stop a model already serving, since inference reads no dataset. So for every m ∈ A that is deployed, hosted or held by a supplier, a named owner records an interim decision at once: hold (out of service) or constrain (running within written limits), with the reason and a review date. The course default is hold; lifting a hold needs a named person, and a hold past its review date with no decision decays to deny, never to allow. Models whose lineage is unknown are treated as in A until shown otherwise.
  2. Notify. The maintainers, whoever policy and law require, and every downstream recipient of a copy or model (for example a supplier). Carlini et al. notified the maintainers of each affected dataset and notified Wikipedia.5
  3. Remove. Publish d@v' with the items removed, a new manifest and hashes; purge every copy and derived training set; verify downloads against v' hashes.65
  4. Retrain or retire. For each m ∈ A, including hosted and supplier copies, retrain from v' and a clean starting point not in A, or retire it.36
  5. Record. Datasheet erratum, new version, and how the old version's obsolescence is communicated; a new model card version for every retrained model.12

Log each step as it happens; step 5 closes the incident, it is not the first write.

Completeness. Computing A is only a query if section 2's linkage exists for both edge types. The counterexample to a data-only query: d@1 → B (base model trained on bad data), then c@1, B → F (fine-tune on a clean dataset). F's card names only c@1, which does not descend from d@1, so a query over training-data fields misses F, though its weights descend from B. With the starting-model field, F ∈ A. Without name@version, manifest hashes and starting-model links in the cards, A is unknown, and removal cannot be shown to be complete.

7. Three yeses, as checks

The data parts of the three yeses (module 100.9): Get to Yes · Stay at Yes · Recover to Yes. A passing check is evidence, not permission; a named person gives each gate answer (allow, constrain, hold or deny). No model, CI job or documentation generator releases a dataset or a model: the checks below can only fail a release or route it to that person.

Receipts. A receipt of a gate decision holds no private content. Example fields: decision, decided_by, time, dataset (name@version), manifest_sha256, model_card (name@version). Never item contents, customer names, service-note text, or the affected items, their URLs or their hashes. The incident log and datasheets live in access-controlled records, not in the receipt. A reference or hash in a receipt is not a licence to disclose what it points at, and a hash of a small or guessable record should not be assumed to be anonymous. At this module's stage the receipt can be an unsigned decision record; modules 400.8 and 400.9 cover signing and making receipts verifiable by a stranger offline, and what such a receipt does not prove.

Terms: datasheet, model card, provenance graph, lineage, licence category (permitted use, attribution, share-alike), split-view poisoning, frontrunning poisoning, cryptographic vs. perceptual hash, manifest, erratum, exposed set, hold, receipt.

Further learning (optional, not sources)

Pointers only; nothing here is copied from them and none supports a claim in this module.

Sources

  1. s1 Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford (2018). Datasheets for Datasets. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  2. s2 Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru (2019). Model Cards for Model Reporting. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  3. s3 Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, et al. (2024). A large-scale audit of dataset licensing and attribution in AI. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  4. s6 European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). Standard. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19; OJ text read via Publications Office Cellar DOC_1, EUR-Lex returned empty 202). ↩ ↩ ↩ ↩
  5. s5 Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, et al. (2023). Poisoning Web-Scale Training Datasets is Practical. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  6. s4 David Thiel (author); Jeffrey Hancock (research team head), as the repository record lists them (2023). Identifying and Eliminating CSAM in Generative ML Training Data and Models. Paper. Checked 2026-11-19 by source-checker (Claude Opus 5.5, separate session; 400.3 round 3 re-check 2026-11-19; abstract only, no PDF linked). ↩ ↩ ↩ ↩

Next: the quiz