Level 400: Engineering AI systems · Module 400.3

Try each exercise first, then open its model answer.

Data with a paper trail · Workbook

Three exercises, one for each thing this module teaches. Everything here is made up and set at Big Widget Company. Do not use real names, customers, documents or datasets from your own work. Write short answers. "Unknown" is an honest answer if you also say who you would ask.

Exercise 1 (400.3-a)

A datasheet and a model card for one training set. Big Widget's quality team wants to fine-tune a model that sorts inspectors' defect reports into categories, so an engineer can review them faster. The training set is bw-defect-reports v1: 12,000 reports written by inspectors at two plants between 2025 and 2026. Some reports name the customer whose order was affected. The team also wants to add a public set of labelled product reviews it found on a dataset hub.

  1. Write a short datasheet with six parts: motivation, composition, collection, consent, licence and known gaps. Where you do not know an answer, say so and name who you would ask.
  2. Which two parts, if left blank, should block release? Why?
  3. Write a short model card for the model: model details, intended use, out-of-scope use, factors, metrics, evaluation data, training data, and the model it was fine-tuned from.
  4. A colleague says: "The product reviews are public, so we don't need to document them." What do you say?
Model answer
  1. A good datasheet covers:
    • Motivation: made by the quality team to train a defect-report sorter; who funded it (for example the quality budget, or "unknown, ask the quality manager").
    • Composition: 12,000 defect reports, 2025 to 2026, from two plants; what each report is; whether it is a sample of all reports and whether that sample is representative; known errors or duplicates; that some reports name customers, so it holds sensitive data; that the added product reviews are an outside resource that may change or carry their own restrictions.
    • Collection: how the reports were pulled, by whom and over what period, and whether there was an ethical or privacy review.
    • Consent: the reports are written by inspectors and name customers, so they relate to people. Were inspectors and customers told, did they agree, and can they withdraw? "Internal, so not needed" is not an answer. If you do not know, write "unknown" and name who will find out.
    • Licence: the terms for the team's own reports (internal use only?), and the traced licence for the product reviews, not just the hub label (see Exercise 2).
    • Known gaps: only two plants; dates covered; any languages missing; tasks it should not be used for (for example, judging individual inspectors).
  2. Collection and licence. Without the collection answer nobody can check that the reports match the job. Without the licence answer nobody knows what the team may do with the mixed-in reviews or the model trained on them. A blank is not an answer. "Not known yet", with a named owner and a date, is an answer, but it still holds the release until it is filled. Someone other than the writer checks the datasheet before release.
  3. A good card names who made it, the date, version and licence; intended use (suggesting a category an engineer checks) and out-of-scope use (no automatic rejection of parts, no judging inspectors); factors (two plants, report language, product line); metrics with an estimate of uncertainty, such as confidence intervals, broken down by factor; the evaluation data and its datasheet; and the training data by datasheet name and version, bw-defect-reports v1 plus the reviews. As this course's own rule, not the paper's, it also names the base model or checkpoint the fine-tune started from. It should be updated with each new model version. If the training data cannot be shared, the card still gives basic details of what it covers. The card is the team's claim, not an independent check.
  4. Public says who can see the data. It does not say why it was collected, what is in it, whether people agreed, or what use is allowed. The datasheet paper proposes that every dataset have a datasheet, and a model trained on the data inherits those answers.

Exercise 2 (400.3-b)

Trace the licence. The product-review dataset from Exercise 1 is labelled on the hub as allowing commercial use. You trace it:

  1. Draw the provenance chain from original source to Big Widget's model, one link per hop.
  2. Mark each link that is missing and each pair that contradicts.
  3. What may the team do now, and what does it record? Who decides?
  4. Why should the team not simply trust the hub label? Give one finding from the provenance audit.
Model answer
  1. (1) Original source: the shopping website and its terms. (2) Dataset: its makers, its paper and the licence for the data. (3) Hub copy: the commercial-use label. (4) Big Widget's training set: bw-defect-reports v1 plus the reviews, by version. (5) The model: its card names the training-set version and the model it started from.
  2. Missing: link 1, because the website's terms have not been found. The repository licence covers the code, not the data, so it does not fill any gap. Contradiction: link 3 (commercial use) against link 2 (research use only).
  3. The gate answer for commercial training is hold: wait for a person. It is not allow: the hub label is not a yes, and only a named person can give one. Keep the reviews out of any commercial training run. Record the contradiction and the missing link in the datasheet, and ask Big Widget's legal lead for a decision. The legal lead decides what the terms actually allow and records the decision and its basis. The audit sorted each licence feature by the strictest licence it found, but that was a way of categorising datasets for research, not a ruling on which licence applies: several licences can be alternatives, can all apply, or can cover different parts of a dataset. The team borrows the caution, not the verdict. Holding on the strictest reading until a named person decides is this course's rule. It is not legal advice and not a finding of the audit.
  4. In the 2023 version of the audit, Longpre and colleagues found licences omitted in over 70% of cases and error rates over 50% on widely used dataset hosting sites. On Hugging Face, 66% of the licences they analysed were in a different use category from the original author's, often labelled as more permissive. Data is re-packaged and re-licensed hop by hop, and each hop can drop or change a condition.

Exercise 3 (400.3-c)

Bad data found. Six months later, the maintainers of an image dataset that Big Widget used to train a parts-photo classifier announce that the dataset contained unlawful material. The dataset is published as a list of web addresses that each user downloads. Big Widget holds a downloaded copy, a cleaned training set built from it, and a cache on a shared drive. Two models were trained on it: one in production and one shared with a supplier. The supplier then fine-tuned its copy on its own photos, which have nothing to do with the bad dataset.

  1. A colleague says: "It's a handful of images in millions, so it can't matter." Give two reasons that is wrong. Be exact about what the research shows and does not show.
  2. Put the course's five steps in order, and write what each step means here. Say what happens to the model already in production while the team works.
  3. Which records make step 4 quick? Is the supplier's fine-tune affected, and how would the team find out? What happens without the records?
  4. The dataset is a list of web addresses. Name one way an attacker could plant data in such a dataset, and one defence the researchers proposed, with its limit. What does a hash mismatch tell you, and what does a match not tell you?
  5. Write the receipt for the decision to hold the production model. What goes in it, and what stays out? What happens if the review date passes and nobody looks at the hold?
Model answer
  1. Unlawful content is not made acceptable by being rare (that is this course's framing), and it does not stay in one file. The Stanford study found many hundreds of instances of known child sexual abuse material in LAION-5B, parts of which were used to train the Stable Diffusion models, and its recommendations reach existing models and their hosting. Be exact: the study does not say every detected item was in every model's training data, or that any model memorised or can reproduce it. It shows that every copy and every model trained on the data is exposed, and has to be traced and assessed rather than assumed clean. Poisoned content is placed on purpose, and a tiny share can be enough: Carlini and colleagues showed in 2023 that 0.01% of LAION-400M or COYO-700M could have been poisoned for $60, and earlier research found even minuscule amounts of adversarial noise can cause targeted mistakes. That was a demonstration of what was possible, not a record of an attack.
  2. (1) Stop use and hold: freeze training runs and releases that use the dataset or any copy. Freezing releases does not stop the production model, which is already serving. A named owner decides at once, for the production model and the supplier's copies, whether each is held (taken out of service) or constrained (kept running only within written limits) while it is assessed, and records the decision and the reason. This course's default is hold. Only a named person can lift it, and the record says when they will look at it again. (2) Notify: the dataset's maintainers, the people the company's policy and the law name, and the supplier. (3) Remove: from the downloaded copy, the cleaned training set and the cache, and check new downloads against the cleaned version, for example by hash. (4) Trace, then retrain or retire: both models and anything built on them, including the supplier's fine-tune. Retraining starts from the cleaned data and a starting checkpoint that never saw the bad data; fine-tuning the affected model again is not a retrain. (5) Record: an erratum and new version in the datasheet, how the old version is retired, and updated model cards for every affected model. Keep an incident log from step 1; "record" is last because it closes the incident. Holding the models stops the exposure but is not a return to yes: call it recovered only when the records list every exposed model, including hosted and supplier copies, and what was done to each, and the named owner agrees the list is complete. This order is the course's procedure, not legal advice.
  3. Model cards that name both the training-set version each model used and the model it started from, plus versioned datasheets. Yes, the supplier's fine-tune is affected: it was trained on clean photos, but it was built on top of a model trained on the bad dataset. A search by training data alone misses it; following the "started from" link finds it. Any model whose lineage is not known yet goes on the list as "not known yet" and stays on hold until someone finds out. Without these records, nobody can say which models to retrain or retire. Training on data is expensive and largely irreversible, so deleting rows from a file does not fix a model.
  4. Split-view poisoning: buy an expired domain the list still points to and serve different content, so later downloads differ from what the curator saw. (Frontrunning, timing an edit just before a snapshot of a crowd-sourced site, is the other attack.) Defence: integrity verification, publishing cryptographic hashes of all content so users can check what they received. Limit: the check also rejects harmless changes such as re-encoding, resizing or cropping an image. Reading a hash check: a mismatch means the bytes you received differ from the bytes the curator recorded; it does not say why, since the cause could be an attack or a harmless re-encoding. Reject or quarantine the item, then find out. A match means you received what was recorded; it does not mean the recorded item was lawful, accurate or safe. A hash makes the dataset tamper-evident, not tamper-proof.
  5. In the receipt: the gate answer (hold: wait for a person; the four answers are allow, constrain, hold and deny), which model it covers, who decided, when, the reason in one line, the review date, and references to the evidence, such as the affected dataset versions, manifest hashes and model-card versions. Kept out: the images, any link to or location of the affected items, customer names and the content of any records. A reference points at the team's controlled record; it does not give anyone the right to see that record. Here the receipt can be an unsigned decision record; modules 400.8 and 400.9 show how to make a signed one a stranger can check. If nobody returns to the hold, it decays to deny: the model stays out of service until a named person decides. It never drifts into allow, and a model, a tool or a passing check cannot lift it.

Next module: 400.4 Operating a model: monitoring, drift and rollback