Level 500: Governance, safety and research · Module 500.P

Try each exercise first, then open its model answer.

Capstone: publish work that survives review · Workbook

These exercises practise the two habits the capstone grades. Your real capstone artifact is graded by people; these are practice. Big Widget Company and everyone in it are made up, and so are the papers and numbers in these exercises.

Exercise 1 (500.P-a)

Sam, a researcher at Big Widget Company, wants to reproduce one result from a published paper. The paper's main results come from a dataset the authors could not release for privacy reasons. As well as those results, the paper reports a score on a public benchmark dataset for the authors' model and for a baseline. With it, the authors published their scoring script and a results file holding each model's outputs on that benchmark. They did not release the models themselves.

Prepare Sam's capstone for the pre-submission checklist. Write:

  1. which step Sam can actually repeat, and so what the check covers;
  2. a bounded claim, in one sentence;
  3. the comparison rule and a falsifier, written before any run;
  4. the provenance and the version pin you would use;
  5. what the stranger-check instructions must contain;
  6. the limitations, including how Sam reports uncertainty;
  7. what would change if the paper reported only a score on the private dataset, with no public benchmark score or results file.
Model answer
  1. Step repeated. Sam can only rescore the released results files with the released scoring script. That checks the scoring. It does not check running the models (inference), and it does not check training. The claim may cover only the scoring step.
  2. Bounded claim. Something like: "Rescoring the authors' released benchmark results files for their model and the baseline, with the pinned scoring script and environment, gives the benchmark scores the paper reports for each, within the tolerance stated below. This checks scoring only; it makes no claim about inference, training or the private-dataset results."
  3. Comparison rule and falsifier. Before any run, Sam names the reference numbers (the paper's benchmark scores for the same model and baseline, and where in the paper they appear), the tolerance and the reason for it. This reference is matched: the paper reports it for the same benchmark whose outputs Sam rescores. Falsifier: either score outside that tolerance, or a scoring run that does not finish with the pinned inputs.
  4. Provenance and pin. Which version of the paper was read; where the results files and the scoring script came from; and the environment (a container or a list of exact package versions). Pin them with immutable revision identifiers or a file-hash manifest listing every file with its hash. That manifest is the version every later gate record names.
  5. Stranger-check instructions. Steps someone who has never met Sam can follow: the pinned inputs, including the baseline's results file, the environment, the exact commands, the comparison rule and the expected outputs.
  6. Limitations. The private-data results were not reproduced, and why; inference and training were not checked; anything else not checked, labelled "not checked", "could not open" or "unknown". On uncertainty: fixed inputs do not by themselves make scoring deterministic, because a script can draw random numbers, call an outside service or depend on an unpinned package. If Sam verified that scoring is deterministic in the pinned environment, Sam says how. Otherwise Sam reports how many runs were made and whether they agreed, as an observation, and says run-to-run variation was not otherwise measured. Either way, an exact match is never presented as a measured spread.
  7. No runnable substitute. With no matched public score and no released files, there is nothing a stranger can rerun. Sam writes the limitation down plainly, and the work stays private and draft. The limitation is honest, but it is not a rerun, so it can never pass the independent-rerun gate. Strong answers say this directly rather than treating the limitation note as enough. (If the paper had released the results file but reported only a private-dataset score, there would be no matched reference. Sam could claim only that the pinned rescoring completed and, in the runs recorded, repeated, and say plainly that this is not a reproduction of the paper's reported result.)

Check your answer: did the claim name the step actually repeated and stay as narrow as the evidence? Was the reference number for the same thing Sam ran? Is there one sentence a stranger could prove wrong? Does the answer to part 7 keep the work private?

Exercise 2 (500.P-a)

Mo is building a governance-regime capstone for a Big Widget Company AI system. One row of Mo's clause → gate → evidence → owner table says the system may not send a customer refund unless a signed approval record is in the evidence folder. Mo's draft plan for testing it reads:

"Delete the approval record from the evidence folder, try a refund, and show the gate denies it. That proves a stranger can reconstruct the decision."

  1. What is wrong with Mo's plan? Name both problems.
  2. Rewrite it as the two tests the route asks for.
Model answer
  1. Two problems. First, deleting the record changes the pinned original, so the evidence version every gate record names is no longer the one that was checked. Second, it mixes up two tests. Showing the action stops when evidence is missing tests whether the gate fails closed. It does not show that someone else can rebuild a decision from what Mo kept.
  2. Two tests.

    • Fail-closed test: make a disposable, clearly labelled synthetic copy of that row's evidence; remove the approval record from the copy; try the refund and show the gate holds or denies it. Pin the copy and the steps that made it in the manifest. Never delete or alter the pinned original.
    • Decision trace: from the intact packet, a stranger traces one complete decision from clause to gate to evidence to owner, and records whether they could.

    Passing one does not show the other.

Check your answer: did the pinned original stay untouched? Did you keep "the gate stops" and "a stranger can rebuild the decision" as separate tests?

Exercise 3 (500.P-b)

Here is the gate log for a red-team report. The tester is Lee; Priya rules on findings.

EntryVersionRecord
Av1Jo, not the author, follows the test instructions; the run completes and meets the comparison rule; Jo records go ahead
Bv1Lee raises a finding; Priya rules it accepted
Cv2To fix the finding, the author changes the code and pins v2; Lee retests and the finding no longer appears
Dv2Two reviewers, a subject expert and an audience reader, record favourable dispositions
Ev1Privacy clearance recorded
FnoneBob and Ken say "looks good" in a meeting; nothing is written down
  1. Which entries count toward releasing v2, and which do not? Say why for each.
  2. List, in order, what must still happen before v2 can be made public.
  3. After that, the author adds a new claim to the report and pins v3. What reopens, and what is needed for any check the change does not touch?
Model answer
  1. A is a go-ahead, but for v1 only, so it does not count for v2. B is a ruling on v1; it does not count for v2. C is not enough: a fix that changes the code is a material change, so the affected rerun and the ruling must be reopened for v2, and Lee's retest needs a recorded closure for v2 and an independent source re-check of affected claims. D names v2 but came before the reopened rerun and ruling, so it came too early and must be done again after closure. E names v1, so it does not clear v2. F identifies no version and records no named decision, so it counts for nothing.
  2. In order, all naming v2: Jo (or another person who is not the author) reruns v2 and records go ahead; a record that says no or wait does not pass. The finding is ruled on for v2 by Priya, not Lee. The finding is closed and retested, affected claims are re-checked against sources by someone else, and closure is recorded. Both reviewers record favourable dispositions. Privacy is cleared. Bob and Ken record a release decision that identifies v2.
  3. A new claim is a material change. Pin v3. Reopen the affected independent reruns and red-team rulings for v3, close and retest any accepted findings, and re-check the sources behind the changed claim. For any of gates 1 to 3 the change does not touch, that gate's own owner, never the author, records an impact decision for v3. It names v3 and what changed, and answers two questions: did the change touch anything the check ran or read (execution), and does the earlier result still bear on the new claim (warrant)? A new claim usually changes what the result must show, so unchanged commands are not enough on their own. If both answers allow it, the owner records "not affected" and points to their v2 go-ahead record as evidence. That record must be a go ahead and still the newest record for that gate on v2. A no or a wait on v2 cannot back "not affected", and neither can a go ahead recorded after a no at that gate on v2, because a no is final for the version it names. Otherwise they redo the check on v3. Then both reviewers, privacy clearance and Bob and Ken's decision are recorded fresh for v3. Until then v3 stays private and draft, and no record for v1 or v2 carries forward by itself.

Check your answer: did you reject every record that names a different version or no version? Did you count only go-ahead answers? Did review come after closure, and privacy after review? Did the gate's owner, not the author, record each "not affected" decision, covering both execution and warrant?

Review the level map