Level 500: Governance, safety and research · Module 500.R

People from different jobs, one in a lab coat, compare notebooks at a shared table.
Who checks the checker?

Research and governance roles (role tracks) · Standard gear

Reviewed and published, version 1.0.0. Big Widget Company, its people and the seven packet cards are made up. The cards are course-owned teaching examples, not real records and not conformance tests. The six-part handoff, the seven role tracks and the prep-rail mapping are this course's own practice, not a law, a standard or an exam syllabus. Where we do not know something, we say so.

What this module does

A receipt carries: hashes, decision, tier, policy hash, key id; A receipt never carries: prompts, outputs, payloads, personal data
The receipt's allowed metadata versus the private content it never carries; packet details belong in a separately controlled work record, and the handoff lists them by hash (s7).
Figure data as text

Keep the evidence, not the secrets

PartWordingSources
A receipt carrieshashes, decision, tier, policy hash, key id[s7]
A receipt never carriesprompts, outputs, payloads, personal data[s7]

The receipt's allowed metadata versus the private content it never carries; packet details belong in a separately controlled work record, and the handoff lists them by hash (s7).

Modules 500.1 to 500.9 each ended with a piece of expert work: a critique of a paper, an evaluation reading, a receipt design, a governance table, a red-team finding. This module asks what happens next. Expert work helps an organisation only when someone else can pick it up, check it and act on it. The record that makes that possible is a handoff.

You take one of seven role tracks: researcher, auditor or assessor, regulator or policy staff, red-team lead, standards editor, governance lead, or reviewer. You work from a small made-up packet card in this lesson (Cards A to G, below). You do not start a new literature review or a new regime design.

By the end you will be able to:

Two beliefs this module corrects:

Why a handoff, and why to someone else

Three of this module's sources, from different fields, point the same way.

Independent checking is a design choice, not an afterthought. In the NIST AI RMF, the Measure function says processes for independent review "can improve the effectiveness of testing and can mitigate internal biases and potential conflicts of interest".1 Subcategory MEASURE 1.3 asks that internal experts who did not serve as front-line developers for the system, and/or independent assessors, are involved in regular assessments and updates.1 The framework's figure of AI actors notes that, as a best practice, those building and using models are separated from those verifying and validating them.1

Auditors who share the organisation's interest need a record others can scrutinise. Raji and colleagues propose SMACTR, an internal audit framework whose stages each produce documents.2 Their Reflection stage ends in a remediation and risk mitigation plan, an algorithmic design history file that should assist an audit trail by "enabling the reconstruction of key decisions and events", and an audit summary report put "in one accessible location for review".2 They also say that because audit results can be ambiguous, "it is critical to identify key stakeholders and decision makers who can drive appropriate responses to audit outcomes".2 And they name the limit plainly: internal auditors "necessarily share an organizational interest with the target of the audit", and keeping an independent viewpoint is challenging.2 They describe internal audits as complementing external accountability, producing artifacts third parties can use.2

Research claims get stronger when someone else reruns them, and the answer is rarely one word. Pineau and colleagues define reproducibility as obtaining similar results to those presented in a paper or talk, using the same code and data when available, and call it "a necessary step" to verify the reliability of findings.3 The NeurIPS 2019 reproducibility challenge aimed to provide independent verification of the empirical claims in accepted papers and to leave a public trace of the findings.3 Participants claimed 173 papers for reproduction.3 The authors report that a reproducibility study rarely gives a concise answer to "Is this paper reproducible?", and that they had not found that the findings of most reproducibility studies lend themselves to a coarse "reproducibility score".3

Put together: a check is stronger when the person checking did not do the work, when the work leaves a record that supports the reconstruction of what was decided, and when the result keeps its nuance instead of being squeezed into a verdict. The handoff is the course's name for that record.

Where the handoff sits in the three yeses. The Foundation names three movements: Get to Yes, Stay at Yes, Recover to Yes.4 Applied to a handoff (the course's application, not a quotation):

The six parts of an assurance handoff

This is the course's own template. Every track uses it.

PartWhat it saysFails if
1. Packet and versionExactly which evidence you received, pinned by a version id or file-hash listSomeone else cannot tell which version you looked at
2. What you checkedThe steps you actually took, and the steps you did notIt lists only what was checked, so coverage looks complete
3. Bounded confidence statementA claim, its scope, the evidence that moved your confidence, what it does not show, and what result would change your mindIt states confidence with no evidence, no scope or no falsifier
4. Unresolved findingAt least one open question or finding, described so it can be ruled on, and marked openIt is quietly closed by the author, or vanishes from the handoff
5. Check instructionsPinned inputs, steps and the expected result, so a stranger can rerun your key checkThe check needs the author present, or private access the next owner lacks
6. Named independent next ownerA named person who did not produce the work being checked and who has the authority to decide, with what they must decide and by whenThe owner is the author, a team name, "the governance process", or someone with no power to act

A few rules make the template hold.

The confidence statement is bounded on four sides. A useful form is: "On version V, within scope S, evidence E supports claim C. It does not show N. If check K returned R, I would withdraw or narrow C." Each blank is checkable. "We are confident the model is fair" is not, because it has no scope, no evidence and no falsifier. The NIST AI RMF says measurement processes should include testing and assessment methods "with associated measures of uncertainty" and "formalized reporting and documentation of results".1

The falsifier tests the claim you made, not a different one. K must be a check whose result bears on C itself. A check that would help explain an open finding belongs with that finding, not in the confidence statement.

Evidence must change confidence, and open questions must stay open. If a piece of evidence would not change your view whichever way it came out, it does not belong in the confidence statement. If a question is still open, write it as open. Do not let a strong piece of evidence on one question close a different question. "I don't know yet" is a fair answer in a handoff. The Foundation pairs it with a person: it "always pairs you with someone".5 Here, that someone is the named next owner.

Independence is this course's rule, with limits you should name. In this course, "independent" means the next owner did not build, test or write the thing being handed over. That rule borrows NIST's best practice of separating building from verifying.1 It is the course's design, not a requirement any source states for handoffs. It removes the author's own interest in the result. It does not remove the interest a colleague shares with the whole organisation: Raji and colleagues' limit on internal auditors still applies.2 They warn against audits "becoming simply acts of reputation management for an organization".2 The course's reasoning, not theirs, is that letting the author quietly close their own finding is one easy route to that. When the shared interest could sway the ruling (a purchase the organisation wants, a launch date it has announced), say so in the handoff, and name an outside assessor if one is available. If none is, record the conflict as unresolved.

A fix and a ruling are different jobs. The author may be the right person to make a fix. The ruling on whether the fix closes the finding goes to someone else.

The owner is a person who can act. Raji and colleagues ask for identified decision makers who can drive a response.2 The NIST AI RMF asks that executive leadership take responsibility for decisions about AI risks, and that responsibilities be assigned and understood for superseding, disengaging or deactivating systems that perform inconsistently with intended use.1 An agent may draft a handoff. Only a person rules on a finding, owns the next decision or signs: this course's own rule is that agents never count as reviewers and only people sign (TRUTH-STANDARD.md, rule 5).

Keep the evidence, not the secrets. If your handoff points to a signed receipt, the receipt holds no private content.4 The Foundation's receipt carries "hashes, decision, tier, policy hash, key id". It never carries "prompts, outputs, payloads, personal data".5 Prompts, transcripts and customer details in a packet stay in a separately controlled work record, if your workplace allows you to keep them. The handoff lists them by hash. A hash does not give you permission to share what it points to.

The packet cards (made up)

The earlier Level 500 modules plan their own course-owned synthetic packets. As of 19 November 2026, the runnable packets and verifier for 500.6 and 500.7 are not yet in this course's repository, so this lesson does not depend on them. Each track below has a small packet card instead. Pin your packet as "500.R, version 0.1.0, Card X". Every key check on a card can be done on paper, by anyone, with only the card. Where a card gives a result that a reader cannot rerun (a signature check, a chatbot retest), the card says so, and your handoff must list it under "not checked".

Card A · Researcher. Big Widget's research group must say whether a published defect-detection result can support a purchase. The paper reports 91.0% accuracy on its released test set, with a spread of ±1.0 points across its own training seeds. Dana, a research scientist, logged six runs of the released code at one pinned commit:

RunDataSeedAccuracy
1Released test set190.6%
2Released test set291.2%
3Released test set390.9%
4Big Widget frozen sample (200 images)183.5%
5Big Widget frozen sample (200 images)284.0%
6Big Widget frozen sample (200 images)383.0%

Key check: average runs 1–3 and runs 4–6. Expected: 90.9% and 83.5%. Cannot be rerun here: the code and images themselves are not supplied.

Card B · Auditor or assessor. Control claim: "Every refund above $500 waits for a person's approval." Each such refund needs a signed decision record and an approval record naming the approver. The pilot-week register lists four refunds above $500:

RefundSigned decision recordVerifier result (as given)Approval record
1PresentValid under key K1Present
2PresentValid under key K1Present
3PresentValid under key K1Absent
4AbsentNonePresent

Key check: count the refunds with both records. Expected: 2 of 4. Cannot be rerun here: the signature verification (the card gives only its result), and whether other refunds above $500 happened but were never registered.

Card C · Regulator or policy staff. Made-up "Widgetland AI Rule 4": "A provider of a hiring AI system must keep a test log. A deployer that substantially modifies a hiring AI system is treated as its provider." Facts: Big Widget bought a hiring assistant from a vendor and uses it. Whether Big Widget changed the assistant's scoring settings: not known. Key check: apply the rule's two sentences to the facts. Expected: whether Rule 4's test-log duty falls on Big Widget turns on the missing fact.

Card D · Red-team lead. Finding F-1: Sam, a tester, got version 4 of Big Widget's support chatbot to reveal another customer's order status. Morgan, the red-team lead, ruled F-1 accepted. A fix was made. Sam retested five listed prompts on version 4, and the log records a refusal for all five. Key check: match the five listed prompts to the five log rows. Expected: five of five recorded as refusals. Cannot be rerun here: the prompts against the version-4 build. Note: the retest was run by the original tester.

Card E · Standards editor. Made-up "Widget Record Format, version 0.3, clause 5": "The record carries the decision time." Test vector V1: a gate decided at 10:00:00, the record was signed at 10:05:00, and the record's time field says 10:05:00. Key check: apply each reading to V1. Reading 1 ("the time the gate decided"): V1 fails. Reading 2 ("the time the record was made"): V1 passes.

Card F · Governance lead. Proposed use: the chatbot approves refunds above $500 with no person in the loop. The gate's answer was hold ("wait for a person"). The required evidence was: (1) a retest of F-1 by someone other than the tester, (2) a working appeal route for customers, (3) a named owner who can switch the feature off. Present: item 3 only. Key check: compare the required list with what is present. Expected: items 1 and 2 missing.

Card G · Reviewer. A draft summary of Card A makes three claims: (1) "The released code reproduces the paper's figure on the released test set." (2) "The model will reach 91% on Big Widget images." (3) "The drop is caused by label noise." Key check: compare each claim with Card A. Expected: claim 1 supported; claim 2 contradicted by runs 4–6; claim 3 not shown by anything on the card.

A worked handoff (researcher track, Card A)

A weak handoff: "I reproduced the paper. I'm fairly confident it works. There may be some issues with our data. Over to the team."

It names a question but cannot be checked. It does not say which version, which data or which steps. "Works" has no scope. The open issue is vague. "The team" is not an owner.

A strong handoff:

  1. Packet and version. 500.R, version 0.1.0, Card A: the paper version, one pinned code commit, the released test set and the 200-image frozen Big Widget sample, as logged.
  2. What I checked. The six logged runs: three seeds on each data set. Not checked: label quality on our sample, other camera types, the paper's ablations, and the precision of the 83.5% figure (no sampling interval was computed for the 200 images).
  3. Bounded confidence statement. On the pinned commit and the released test set, runs 1–3 average 90.9%, inside the paper's 90.0–92.0 band. This supports C: "the released code produces the reported result on the released data". It does not show that the model will reach that figure on Big Widget images; runs 4–6 average 83.5%. Falsifier for C: if a rerun by someone else, at the same commit on the released test set with the same three seeds, averaged outside 90.0–92.0, I would withdraw C.
  4. Unresolved finding (open). The 7.4-point drop on our sample is unexplained. What the numbers do and do not say: the paper's ±1.0 is run-to-run spread across seeds on the released set. It tells you how much a rerun there should wobble. It is not a test of a difference between two data sets. The drop is far larger than that seed spread, so seed-to-seed wobble of the kind the paper measured is an unlikely explanation on its own. Nothing on the card says why it happened. Candidates: label errors in our sample, a different camera, or the model itself. Not attributed, not ruled on.
  5. Check instructions. With only Card A: average runs 1–3 and runs 4–6; expect 90.9% and 83.5%. No access to Dana's machine is needed. A full rerun would also need the code, the environment file and the images, which this lesson does not supply: listed as not checked.
  6. Next owner. Priya, a data-quality lead who did not run either experiment and who can commission checks, decides by the purchasing review which check to commission first. One option is a controlled relabelling check: a second labeller relabels the same 200 images, while the code, seeds and images stay unchanged. If the gap closed, that would point to label errors on this sample. It would not show label errors are the only cause elsewhere. A camera check is another option. The purchase waits for a person: it holds until Priya rules.

The evidence changed confidence (the reruns moved "reported" to "reproduced on released data"), the falsifier tests that claim and nothing else, the open question stayed open, and the owner did not produce the work. That is the shape every track aims for.

The seven tracks

Pick the track nearest your job. When an earlier Level 500 module's synthetic packet is published and checked, you may use it instead of the card. Until then, use the card.

TrackCardWhat you hand offA typical bounded claimWhat usually stays openTypical independent next owner
ResearcherAA replication discrepancy"Reproduces on the released data; not shown on ours"Why the result movedA data-quality or methods lead who did not run it
Auditor or assessorBA missing-evidence register"Refunds 1 and 2 have both required records; refunds 3 and 4 do not"Whether the control ran on every refund, and workedThe control owner's manager, or an external assessor
Regulator or policy staffCA scope-exception memo"On these stated facts, whether the duty applies turns on one missing fact"The missing role fact; any legal conclusionCounsel or the decision-maker named in the rule
Red-team leadDA ruling and retest handoff"F-1 was accepted; a retest on version 4 recorded refusals for these five prompts"Coverage beyond five prompts; a retest by someone other than the testerA retester other than the original tester
Standards editorEA versioned ambiguity and test-vector note"Clause 5 in version 0.3 can be read two ways; V1 separates them"Which reading the editors adoptThe editors' group, with a named chair
Governance leadFAn escalation record"The gate held this use because items 1 and 2 were missing"Go ahead, only this far, wait, or noThe executive named as accountable
ReviewerGAn evidence and revision disposition"Claim 1 is supported by Card A; claims 2 and 3 are not"The author's revisionThe author makes the fix; a second reviewer rules on the re-read

Notes on the tracks, tied to the sources:

The NIST material also warns against using any of this as a box-ticking list. The AI RMF says its actions "do not constitute a checklist", and it does not prescribe risk tolerance.1 The Playbook is for voluntary use, and organisations may borrow "as many – or as few – suggestions as apply" to their use case.6 The handoff template is a minimum shape for a record, not a measure of whether the work underneath was good.

Four common failures

  1. Confidence without a handle. "High confidence" with no evidence, scope or falsifier. Fix: fill all four sides of the confidence statement, with a falsifier that tests the claim itself.
  2. The quiet close. The author marks their own open finding resolved. Fix: open findings stay open until the named next owner rules.
  3. Handing back to yourself. The next owner is the author, the author's direct collaborator on the same work, or a team. Fix: name a person who did not produce the work and who can decide.
  4. Instructions only the author can run. The check needs the author's machine, memory or private access. Fix: pin inputs, list steps and expected outputs, and test that a colleague can follow them. Whatever a stranger cannot rerun goes under "not checked".

Mapping your handoff to the prep rail

Level 500 includes one practice rail toward an AI governance or management-system audit certification. It is this course's own practice, not an exam. Its structure comes from this course's Level 500 map (levels/500/objectives.yml), not from any source cited here. On Bob's decision of 18 November 2026, the vendor and exam stay unnamed until Bob clears each name. No current exam alignment is claimed, and this course supplies no vendor-proctored exam and no endorsement.

The rail draws on six course objectives: 500.6-a, 500.7-b, 500.8-b, 500.9-d, 500.R-a and 500.R-b. Its practice is to produce a role-specific handoff, inspect a missing-evidence packet, and route an unresolved finding to an independent owner. You map each practice record to the objectives it exercises:

Handoff partRail practice it exercisesCourse objective it draws on
Packet and version; bounded confidence statementProduce a role-specific handoff500.R-a
What was checked; missing-proof listInspect a missing-evidence packet500.6-a (integrity and origin versus attestation and effectiveness); 500.7-b (missing evidence for a control claim)
Clause, gate, evidence and owner rowsInspect a missing-evidence packet500.8-b
Tested and untested coverage; ruling by someone other than the testerRoute an unresolved finding to an independent owner500.9-d
Unresolved finding; named independent next ownerRoute an unresolved finding to an independent owner500.R-a
This mapping, with each item's evidence typed against its claimAll three500.R-b

Evidence about a person is not evidence about a system

Sort evidence by the claim you want it to support, not by the item alone. The same item can bear on two claims. Morgan's ruling on F-1 (Card D) shows something about Morgan's practice as a red-team lead. It also records a claim about the chatbot. Ask of each item: which claim is this evidence for, and what does it fail to show?

ItemWhat it can support, under its assumptionsWhat it does not show
Retest log (Card D)The listed prompts were recorded as refused on version 4, if the log is complete and genuineOther prompts, other versions, or a retest by someone other than the tester
Signed record that verifiesThe record is unchanged since someone holding the key signed it, if the key was not misused and the verifier is correctThat the event it describes happened, or that the control worked
Ruled red-team findingA weakness was found and a person ruled on itThat no other weaknesses exist, or that the system is ready overall
Complaint and appeal recordsAn appeal route exists and was usedThat appeals were decided well, or that everyone could reach the route
Register with gaps (Card B)Which required records are present and which are absentWhether the control ran on the cases that were never registered

One rule does not bend: a person's exam result is never evidence about a system's controls.

The NIST AI RMF keeps people and systems apart too. GOVERN 2.2 is about people: personnel and partners receive AI risk management training to perform their duties.1 MEASURE 1.3 and MEASURE 2.1 are about the system: who assesses it, and documenting the test sets, metrics and tools used.1 The framework's trustworthiness characteristics (valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; fair with harmful bias managed) are characteristics of AI systems, not of the people who assess them.1

So:

Check yourself

  1. Your handoff says "The vendor's model is fair, high confidence." Name the three things missing from its confidence statement.
  2. You are the red-team lead on Card D. The tester who found F-1 also ran the retest. Who must be the next owner of the retest result, and why?
  3. A colleague says the rail mapping table "proves our hiring assistant is audit-ready". Which claim is that table evidence for, and what is wrong with the colleague's claim?

Further learning (not a citation)

These free university courses go deeper. They are not sources for any claim in this module, and their materials are not copied here: Harvard CS 2881R, AI Safety (Fall 2025); Stanford CS 181/181W, Computers, Ethics, and Public Policy (Spring 2024); Stanford CS25, Transformers United V6. See levels/500/universities.md for links, and note its warnings about what each course does not teach.

Glossary

Sources

  1. s2 Tabassi, Elham (National Institute of Standards and Technology) (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Standard. Checked 2026-11-19 by source-checker agent (Claude, fresh session; not the writer, reviser, item writer or an earlier checker), 19 Nov 2026, round 3 re-check. ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  2. s3 Raji, Inioluwa Deborah; Smart, Andrew; White, Rebecca N.; Mitchell, Margaret; Gebru, Timnit; Hutchinson, Ben; Smith-Loud, Jamila; Theron, Daniel; Barnes, Parker (2020). Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. Paper. Checked 2026-11-19 by source-checker agent (Claude, fresh session; not the writer, reviser, item writer or an earlier checker), 19 Nov 2026, round 3 re-check. ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  3. s5 Pineau, Joelle; Vincent-Lamarre, Philippe; Sinha, Koustuv; Larivière, Vincent; Beygelzimer, Alina; d'Alché-Buc, Florence; Fox, Emily; Larochelle, Hugo (2021). Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program). Paper. Checked 2026-11-19 by source-checker agent (Claude, fresh session; not the writer, reviser, item writer or an earlier checker), 19 Nov 2026, round 3 re-check. ↩ ↩ ↩ ↩ ↩ ↩ ↩
  4. s6 AiGovOps Foundation (n.d.). AiGovOps Foundation Manifesto. Official doc. Checked 2026-11-20 by source-checker agent (Claude, fresh session; not the writer, reviser, item writer or an earlier checker), 20 Nov 2026, round 7 re-check. ↩ ↩ ↩ ↩
  5. s7 AiGovOps Foundation (n.d.). AiGovOps Foundation Practice (home page). Official doc. Checked 2026-11-20 by source-checker agent (Claude, fresh session; not the writer, reviser, item writer or an earlier checker), 20 Nov 2026, round 7 re-check. ↩ ↩
  6. s1 National Institute of Standards and Technology (2026). NIST AI RMF Playbook. Official doc. Checked 2026-11-20 by source-checker agent (Claude, fresh session; not the writer, reviser, item writer or an earlier checker), 20 Nov 2026, round 7 re-check. ↩ ↩

Next: the quiz