
Research and governance roles (role tracks) · Plain gear
Reviewed and published, version 1.0.0. Big Widget Company, the people in it and the seven packet cards are made up. The six-part handoff, the seven tracks and the practice rail are this course's own way of working. They are not a law, a standard or an exam. Where we don't know something, we say so.
What this part is for
Figure data as text
Keep the evidence, not the secrets
| Part | Wording | Sources |
|---|---|---|
| A receipt carries | hashes, decision, tier, policy hash, key id | [s7] |
| A receipt never carries | prompts, outputs, payloads, personal data | [s7] |
The receipt's allowed metadata versus the private content it never carries; packet details belong in a separately controlled work record, and the handoff lists them by hash (s7).
In the last nine modules you did expert work. You checked a paper. You read a safety test. You designed a receipt, a set of rules and a red-team exercise.
Now a hard question: what happens to that work when you hand it on?
Think of a relay race. You can run your leg very fast. But if you drop the baton, the team loses. In this module, the baton is a handoff: a short record that lets someone else check your work and decide what to do next.
You pick one of seven tracks: researcher, auditor, policy staff, red-team lead, standards editor, governance lead or reviewer. Each track has a small made-up packet in this lesson, called a card.
When you finish, you can do two things:
- Write a handoff that someone else can check. It says how sure you are and why. It keeps the open question open. It names who checks next.
- Explain how your practice links to this course's practice rail. And explain why proof that you are ready is not proof that a system is safe.
Two wrong ideas this part fixes:
- "An expert saying 'I'm confident' is enough." It isn't. If nobody else can check it, it is just an opinion. A US government guide on AI risk, from NIST, asks that people's jobs in AI risk work are written down. It asks that they are clear to people and teams across the whole organisation.1
- "Finishing a practice course certifies me, or the system." It certifies neither. This course gives no certificate, and proof about a person is not proof about a system.
Why hand off to someone else?
Picture a chef who tastes their own soup and says, "Perfect." Maybe it is. But a second taster, who did not cook it, is more likely to notice the salt.
The sources say something close to this about AI work.
- NIST says that checks by someone independent can make testing better. They can also cut down on bias and conflicts of interest.1 NIST also says, as a good habit, to keep the people who build a model apart from the people who check it.1
- Raji and colleagues wrote a guide for auditors inside a company. Each step makes a document. At the end there is a file meant to help with "the reconstruction of key decisions and events", and a summary report in one place for review.2 They also warn that auditors inside a company share the company's interest, so staying neutral is hard.2
- Pineau and colleagues report on a program at a big AI research conference in 2019. People tried to rerun the results of published papers, and 173 papers were picked for this.3 They found the answer to "Can this paper be rerun?" was rarely a simple yes or no.3
So a good check has three things. Someone else does it. There is a record they can follow. And the answer keeps its "it depends" parts instead of hiding them.
The three yeses. The Foundation names three steps: Get to Yes, Stay at Yes, Recover to Yes.4 Here is how a handoff fits them. This is the course's own way of using them.
- Get to Yes: before anyone acts on your work, the handoff says what it shows, what it doesn't, and who decides. A handoff is not a yes by itself.
- Stay at Yes: the open question stays open, with a named person. A new version gets checked again.
- Recover to Yes: after a problem is found, someone fixes it, someone else retests it, and a person rules. Saying "fixed" is not proof.
The six parts of a handoff
Every track uses the same six parts.
- What you were given. Name the exact version of the evidence, so nobody mixes it up with another version.
- What you checked, and what you didn't. Both lists. If you only list what you checked, it looks like you checked everything.
- How sure you are, with limits. Say what you think is true. Say where it holds. Say what evidence changed your mind. Say what it does not show. Say what result would prove you wrong. That last test must be about the thing you claimed, not about something else.
- The open question. At least one thing is still not settled. Write it down and mark it open. You do not get to close it yourself. "I don't know yet" is a fine answer here. The Foundation says it "always pairs you with someone".5 Part 6 names that someone.
- How to check it. Steps a stranger can follow, with the files they need and the answer they should get. They should not need you in the room. If they can't redo some part, list it as "not checked".
- Who checks next. A real person, by name. It must be someone who did not do the work, and who can actually decide. Say what they must decide.
"How sure you are" is the tricky one. "We're very confident it's fair" says nothing a person can check.
NIST says testing should come with a measure of how uncertain the result is, and with written reports.1
Some rules that hold it together.
- Someone else is this course's rule. It borrows NIST's habit of keeping builders and checkers apart.1 It stops you judging your own work. It does not stop a colleague caring about the company's good name. Raji and colleagues warn that audits can turn into "reputation management".2 If that could sway the decision, say so. Name an outside checker if there is one.
- Fixing and deciding are two jobs. You may fix your own mistake. Someone else decides if the fix worked.
- Only people decide. An AI may help draft a handoff. A person rules, owns the next step and signs. That is this course's rule.
- Keep secrets out. A Foundation receipt holds no private content.4 It never carries "prompts, outputs, payloads, personal data".5 Keep chat logs and customer details in a separate, controlled record. Your handoff lists them by fingerprint (a hash).
The packet cards (made up)
Each track gets one small card. Everything you need is on the card. You can do the key check with a pencil. The earlier Level 500 modules plan their own made-up packets, but on 19 November 2026 the ones for 500.6 and 500.7 were not yet in this course's files. So use the card. Name it like this: "500.R, version 0.1.0, Card A".
- Card A · Researcher. A paper says its model scores 91.0% on its own test photos. Its runs vary by about 1 point either way. Dana reran the code three times on the paper's photos: 90.6%, 91.2% and 90.9%. She ran it three times on 200 of Big Widget's photos: 83.5%, 84.0% and 83.0%. Check: work out each average. You should get 90.9% and 83.5%.
- Card B · Auditor. Rule: every refund over $500 waits for a person to approve it. Each one needs a signed record and an approval record. Four refunds: 1 and 2 have both. 3 has no approval record. 4 has no signed record. Check: count the complete ones. You should get 2 of 4. You can't redo the signature check yourself; the card only gives its result.
- Card C · Policy staff. A made-up rule says the maker of a hiring AI must keep a test log. A user who changes the AI a lot counts as its maker. Big Widget bought one and uses it. Did it change the settings? Nobody knows. Check: read the rule against the facts. Whether the duty applies depends on that missing fact.
- Card D · Red-team lead. Sam found that the chatbot (version 4) would show another customer's order. Morgan, the red-team lead, accepted the problem. It was fixed. Sam retested five prompts, and the log shows a refusal for all five. Check: match the five prompts to the five log lines. Note: the person who found it also retested it. The log tells you what was recorded in Sam's run. It does not prove the record is complete or labelled right. That is one check: compare the log with Sam's original prompts and replies. A later run that goes differently does not, on its own, prove the log wrong. It asks a new question: does the fix hold every time? That is a second check. It needs its own run, on the same pinned version, with the rule for "holds" written down first.
- Card E · Standards editor. A made-up rule says, "The record carries the decision time." Example: the gate decided at 10:00, the record was signed at 10:05, and it says 10:05. Check: if "decision time" means when the gate decided, the example fails. If it means when the record was made, it passes.
- Card F · Governance lead. Someone wants the chatbot to approve big refunds with no person involved. The gate said "wait for a person". It needed three things: a retest by someone who isn't the tester, a way for customers to appeal, and a named person who can switch it off. Only the third is there. Check: compare the lists. Two are missing.
- Card G · Reviewer. A draft about Card A says: (1) the code matches the paper on the paper's photos; (2) it will score 91% on Big Widget photos; (3) bad labels caused the drop. Check: compare each with Card A. 1 holds up. 2 is wrong. 3 is not shown.
A made-up example (Card A)
A weak handoff: "I reran it. Pretty sure it works. Maybe some issues with our data. Over to the team."
What's wrong? Which version? Which photos? "Works" where? What issues? And "the team" is not a person.
A strong handoff:
- Given: 500.R, version 0.1.0, Card A.
- Checked: the six runs on the card. Not checked: whether our photos are labelled right, other cameras, and how exact the 83.5% is with only 200 photos.
- How sure: on the paper's photos, the average is 90.9%. That is inside the paper's range. So I can say "the code does what the paper says on the paper's photos". I can't say "it will work on our photos". What would prove me wrong: if someone else reran it on the paper's photos and the average fell outside 90 to 92, I'd take my claim back.
- Open question: why is it 7.4 points worse on our photos? The paper's "1 point either way" only tells you how much reruns on its own photos wobble. It can't tell you why our photos score lower. Maybe the labels. Maybe the camera. Maybe the model. Not decided.
- How to check: use Card A. Work out the two averages. You should get 90.9% and 83.5%. You don't need Dana's laptop. A full rerun would need the code and photos, which aren't supplied, so that is "not checked".
- Who checks next: Priya, who looks after data quality and did not run either test. She can order new checks. She decides which check comes first. One choice: have someone else relabel the same 200 photos and rerun, changing nothing else. If the gap closed, that would point to the labels on these photos. It wouldn't prove labels are the only cause. Nobody buys the model until she decides.
The seven tracks
| Track | Card | Work you hand on | What usually stays open | Who might check next |
|---|---|---|---|---|
| Researcher | A | A result that didn't rerun the same way | Why it changed | A methods lead who didn't run it |
| Auditor | B | A list of missing evidence | Did the control run every time, and work? | The control owner's boss, or an outside auditor |
| Policy staff | C | A note on what rules may apply, and what facts are missing | The missing facts | A lawyer, or the person named to decide |
| Red-team lead | D | A ruling on a problem, and a retest | What nobody tested; a retest by someone new | A retester who didn't find the problem |
| Standards editor | E | A note that one rule can be read two ways, with an example that tells them apart | Which reading wins | The editors' group, with a named chair |
| Governance lead | F | A record of why a gate said "wait" | Go ahead, only this far, wait, or no? | The leader named as responsible |
| Reviewer | G | A note on which claims hold up and which don't | The author's fix | The author fixes it; a second reviewer decides |
When an earlier module's made-up packet is ready and checked, you may use it instead of the card.
Some tips from the sources:
- Reviewers: at that 2019 conference, about a third of reviewers who were asked (34%) said a checklist from authors helped them judge the paper.3 The report's authors also said they did not yet have firm proof that their program made papers better.3 Be just as careful. Say which claims hold up. Don't just say "good paper".
- Governance leads: a gate answers "go ahead · only this far · wait for a person · no".4 Write down which answer it gave, and who can change it. A "wait" is not a quiet yes. The Foundation's checklist asks you to name the person who lifts a hold, and how long a hold lasts before it turns into a no.4 The checklist does not say a hold is the same as a wait. That link is this course's own reading: we treat a wait as a hold. This course also adds its own rule: an AI that drafted the record can't be that person. NIST asks for ways for users to report problems and to appeal results.1 It also asks for plans to watch an AI system after it goes live, including appeals and overrides.1
- Standards editors: rules and guides change. NIST's own guide page was updated on 10 June 2026.6 Always say which version you mean.
One warning. NIST says its steps "do not constitute a checklist".1 Its playbook is voluntary. Groups can take as many or as few of its ideas as fit them.6 The six parts are the least a handoff needs. They don't prove the work underneath was good.
Four ways handoffs go wrong
- "I'm sure." But no reason, no limits and no way to be proved wrong.
- The quiet close. You mark your own open question as done.
- Handing it back to yourself. The next checker is you, your partner on the work, or "the team".
- Steps only you can follow. The check needs your laptop or your memory.
Your practice and the practice rail
Level 500 has one practice rail. It points toward an exam for people who audit AI governance. The rail's shape comes from this course's own plan for Level 500, not from a source. The course does not name the exam or the company that runs it. Bob decided on 18 November 2026 to keep them unnamed until he clears each one. The course does not give the exam and does not claim to match it.
The rail uses six course goals: 500.6-a, 500.7-b, 500.8-b, 500.9-d, 500.R-a and 500.R-b. It asks you to practise three things:
- write a handoff (this module);
- look through a packet and list the missing evidence (500.6, 500.7, 500.8);
- send an open problem to someone independent (500.9 and this module).
For each handoff you write, make a small table. In one column, put the part of your handoff. In the next, put which of the three practices it shows. In the last, put the course goal it links to.
Proof about you is not proof about the system
This is the big idea of this part.
Think about a driving test. Passing it says you can drive. It says nothing about whether the car's brakes work. You still need the brakes checked.
First ask: what claim is this evidence for? The same item can count for two claims. Morgan's ruling on Card D says something about Morgan's skill. It also says something about the chatbot.
- Proof about you: your handoffs, your table, a reviewer's comments on your practice, an exam result if you take one later.
- Proof about the system: its test results, its signed records, ruled red-team findings, records of complaints and appeals.
Even system proof only shows a little at a time:
- A retest log shows those prompts were refused on that version. Not other prompts.
- A signed record that checks out shows the record wasn't changed after signing. It doesn't show the event really happened, or that the control worked.
- A ruled finding shows one problem was found and judged. It doesn't show there are no others.
- Appeal records show people could appeal. They don't show appeals were decided well.
NIST keeps people and systems apart too. One of its steps is about training people.1 Others are about testing the system and writing down how it was tested.1 NIST's list of what makes an AI system trustworthy, such as being safe, secure and fair, is about the system, not the person checking it.1
So:
- an exam result is about a person, never a system;
- a well-trained auditor does not make the system pass;
- no single record shows a system is "ready";
- finishing this rail gives no certificate to you or to any system.
Check yourself
- A handoff says, "The model is fair. We're very sure." What three things are missing?
- On Card D, the person who found the problem also retested the fix. Who should check next, and why?
- Someone says your practice table "proves our hiring tool is ready for audit". What claim is the table proof for? What is wrong with what they said?
Words to know
- Handoff: a short record that lets someone else check your work and decide what to do next.
- Open question: something still not settled. It stays open until the next owner decides.
- Independent: did not do the work being checked. That is not the same as not caring about the company.
- Packet card: a small made-up set of evidence in this lesson that you can check with a pencil.
- Rerun (reproduce): getting similar results to a paper's, using the same code and data when you can.3
- Practice rail: this course's practice route toward an exam. It gives no certificate.