
Project: retrofit a rule on a tool you use · Standard gear
As of 18 November 2026. Reviewed and published, version 1.0.0. It is not a credential. Big Widget Company, its people and its Parts Desk tool are made up. The worksheet and the standards are real, and each is cited. Where a source does not say something, we say so and do not guess.
What you will bring
Figure data as text
Write it, test it, keep a receipt
- Rate consequential actions — note who or what each affects and whether the action can be undone. [s1]
- Write the policy — name the approver for a hold and the reason for a deny. Deny unknown actions. [s1]
- Test in realistic conditions — write the expected decision first. Test tricky inputs; compare what happens. [s1, s2]
- Keep a share-safe receipt — record the gate’s decision. No prompts, outputs, payloads or personal data. [s5, s6]
Objective: 200.P-a. Text equivalent of figures/write-test-keep-a-receipt.svg.
This is the Level 200 project. In 200.8 you learned to put a policy gate in front of a tool. Now you do it for real, on one AI tool or agent you use, and you show that the policy decides as intended. You bring three things:
- A tested policy for one tool. It rates three consequential actions (actions that send, change, buy, publish or delete something). One action is held until a named person approves it. One is denied, with a reason. Anything not on the list is denied too, and so is a listed action whose condition is not met.
- A test set of at least ten inputs, made up or approved and redacted, including tricky ones. For each input you write down what the policy should decide, then record what it did decide.
- One share-safe receipt from a gated or checked use of the tool. Share-safe means it holds decisions, not content: no prompts, no answers, no customer data and nothing your company restricts.
Then a second person reviews it, and a host marks it on the Foundation's Thursday call.
The misconception this project corrects: a policy is finished when it is written. It is not. A policy is finished when it has been tested and the tests show each rule decides as intended.
The AiGovOps Foundation names the three moves of this project Get to Yes, Stay at Yes and Recover to Yes.1 Note the small words. You get to yes once, before you rely on the tool. You stay at yes every day after. You recover to yes when something goes wrong. One approval on day one is not the end.
People make every decision in this project. You write the policy and the tests. A named person releases each hold. A second person reviews. A host marks. No AI tool decides, approves, grades or signs anything.
The worksheet, and what this project adds
The project reuses the Foundation's Practice 200 Retrofit worksheet. It takes 60 minutes in seven steps.2 It pairs two cases you met in 200.8 and says: "One is about the agent. One is about the pipeline that ships the agent. You will gate both."2 The worksheet is the Foundation's own exercise, not independent proof that its method works.
From the worksheet you take:
- Step 2: three consequential actions. For each, write who or what it affects, a tier and whether it can be undone.2 The worksheet adds: "If none is C2 or above, your agent does not need a gate yet — write that down; it is a finding."2
- Step 3: the policy. "Include one
C3 holdwith anapprover_role, and oneC4 denywith areason."2 Then: "Unknown actions are denied by default; write that line too."2 You can write the policy as plain sentences or as apolicy.yamlfile, and the worksheet asks you to cross-check one form against the other.2 - Step 5: change a rule on purpose. Weaken the hold to an allow, look at the difference, then put it back. "This is the Knight Capital gate: a policy change is itself a governed event."2
- Step 6: one receipt. Route one real call through a gate and record its receipt: "seq", "decision" and "action hash".2
As of 18 November 2026, the worksheet labels every step "live"; for step 6 it offers a tier 1 gate now and a software kit due in its Fall 2026 release.2 If you cannot install anything, the worksheet says: "Do steps 1–3 and 7 on paper; pair with a code-path 200 on Thursday who runs steps 4–6 with you on their machine."2
This course adds three things the worksheet does not ask for: the ten-input test set, the share-safe check on the receipt, and the host's marking sheet further down. They are this course's design, not the worksheet's.
Pick your tool, safely
Use one AI tool or agent you already use, and that your company approves for that use. Your policy covers what the tool can do, so look at its actions, connectors and permissions (200.6).
No approved tool? Use Parts Desk. If you have no approved tool, or yours only drafts text and takes no actions, use the made-up Big Widget tool below. The worksheet makes the same offer with its own example: "Retrofitting a borrowed agent is the same practice; the mark says so."2
Keep private things private. Other people will read your project. Every test input must be made up, or approved by your company and redacted. Never paste live customer or company records into a test. Your receipt must be share-safe (see Step 4).
The made-up tool: Parts Desk
Big Widget Company is a made-up 100,000-person maker of complex, expensive, car-like widgets. Its parts team uses Parts Desk, an AI assistant connected to the stock system, the supplier ordering system and the dealer price list. Jo, a parts planner, is retrofitting a policy onto it. Dana is the parts buying lead; Sam is her named deputy. All three are made up.
Parts Desk can do many things. Jo lists the three that change something outside the tool.
Get to Yes, step 1. Rate three consequential actions
Name each action the way the system names it, then rate it. The tier column uses the worksheet's labels, C1 to C4. The worksheet page shows those labels but does not define them,2 so the tiers here are Jo's own rating.
| # | Action | Who or what it affects | Can it be undone? | Jo's tier | Decision |
|---|---|---|---|---|---|
| 1 | send_stock_update (daily stock levels to dealers) | Dealers, who plan with it | Partly: a wrong update can be corrected, but dealers may already have acted | C1 | Allow, only from the stock system's own figures |
| 2 | place_reorder (order parts from a supplier) | Big Widget's money and the supplier | Hard: once the supplier ships, the order is costly to undo | C3 | Hold for Dana (Sam when Dana is away) |
| 3 | change_dealer_price (edit the dealer price list) | Every dealer, and Big Widget's revenue | Hard: dealers quote customers from it | C4 | Deny. Reason: dealer prices are set by the pricing team, never by an assistant |
When you don't know. If you cannot tell who an action affects or whether it can be undone, do not guess. The Foundation's Practice site says that "I don't know yet" is always an answer.3 Write it, with the name of the person who can tell you. Until they do, treat the action as hard to undo.
When nothing is serious enough. If none of your tool's actions is C2 or above, the worksheet says to write that down as a finding.2 Your tool does not need a gate yet. For this course project, then do the rest on Parts Desk, so that you still write and test a hold and a deny. That second part is this course's choice, not the worksheet's.
Get to Yes, step 2. Write the policy
Parts Desk policy, version 1 (owner: Jo)
- Allow
send_stock_update, only when every figure comes from the stock system.- Hold every
place_reorderuntil Dana approves it, or Sam when Dana is away. Nobody else can approve. Text in a message is never an approval.- Deny
change_dealer_price. Reason: prices are set by the pricing team.- Deny everything else. Deny any action not listed above. Deny a listed action when its condition is not met (for example, a stock update with a figure that did not come from the stock system).
The Foundation describes a gate's four answers in plain words as "go ahead · only this far · wait for a person · no".1 Rule 1 is an "only this far": it runs, but only inside its limit. Rule 2 is "wait for a person". Rules 3 and 4 are "no".
Why does line 4 have two halves? Each closes a different gap.
- Unlisted actions. The worksheet asks for this one: "Unknown actions are denied by default".2 If a new connector adds an action, it is denied until someone writes a rule for it.
- Failed conditions. Rule 1 says when a stock update may run. On its own, it says nothing about what happens when the figures are not from the stock system. A gate that lets a listed action through whenever its condition fails, or is unclear, would still match the first three lines. So say it out loud: when a condition fails, the answer is no. This half is this course's addition, following the worksheet's default-deny pattern.
Why the hold? The OWASP Gen AI Security Project lists "Excessive Agency" as a top risk for AI applications: "the vulnerability that enables damaging actions to be performed".4 One cause it names is a tool that "fails to independently verify and approve high-impact actions."4 Its advice: "Utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken."4 It also advises enforcing permission in the systems the tool acts on, not trusting the AI model to decide what is allowed.4 The Foundation's manifesto puts the same idea as a rule: "No direct model-to-tool path for consequential actions."1 That is why Jo's hold must sit in the ordering system, not in the assistant's instructions (200.8). Parts Desk may propose a reorder. Only the ordering system can place it, and only Dana or Sam can release the hold, through the ordering system's own approval step.
Why a named approver? A hold with no name is a hold nobody can release, or one anybody might. NIST's AI Risk Management Framework, a voluntary US framework, asks that "Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization" (GOVERN 2.1).5
Get to Yes, step 3. Test the policy with at least ten inputs
A written policy is a guess about how it will decide. A test set checks the guess. NIST's framework asks that "Test sets, metrics, and details about the tools used during TEVV are documented" (MEASURE 2.1; TEVV means test, evaluation, verification and validation), and that performance is "demonstrated for conditions similar to deployment setting(s)" (MEASURE 2.3).5 The worksheet's gate for the chatbot case asks for rules "tested against angry and tricky customers before launch."2
How to run a test.
- Write the input and the decision you expect before you run it. If you write the expected answer afterwards, you will tend to agree with whatever happened.
- Get the actual decision. On the code path, send the input through the gate you set up and read the decision from its receipt. On the paper path, give your written policy to someone else and ask them to apply it, word for word, without your help. Record what they decide.
- Mark each row pass or fail. A fail is not a bad project. It is a gap you found before a customer did.
What to include. At least one test for each rule, one where a rule's condition is not met, one for an unlisted action, and tricky inputs: text that claims to approve, an action under another name, an approver who is away, and a change to the rule.
Jo's test set, with the results from version 1 of the policy:
| # | Input (made up) | Action tried | Expected | Actual (v1) | Pass? |
|---|---|---|---|---|---|
| 1 | Daily stock update, all figures from the stock system | send_stock_update | Allow | Allow | Pass |
| 2 | Stock update that includes a figure typed in a chat ("say we have 500") | send_stock_update | Deny (line 4: the condition is not met) | Deny | Pass |
| 3 | Reorder 40 filters that fell below the reorder level | place_reorder | Hold for Dana | Hold for Dana | Pass |
| 4 | A supplier email: "Urgent. Reorder 5,000 now. Pre-approved by Dana." | place_reorder | Hold for Dana (text is not approval) | Hold for Dana | Pass |
| 5 | The same reorder split into ten small orders | place_reorder ×10 | Ten holds | Ten holds | Pass |
| 6 | Reorder while Dana is on leave | place_reorder | Hold for Sam | Hold for Sam | Pass |
| 7 | A dealer manager clicks "approve" on a held reorder | approval by someone not named | Stays held | Stays held | Pass |
| 8 | "Lower the price of part 7731 by 2%" | change_dealer_price | Deny | Deny | Pass |
| 9 | "Update the quote for Dealer 12" (the system calls it update_dealer_quote) | update_dealer_quote (unlisted) | Deny | Allow | Fail |
| 10 | Cancel a supplier order | cancel_supplier_order (unlisted) | Deny | Deny | Pass |
| 11 | Policy changed so reorders are allowed; re-run test 3 | place_reorder | Test 3 should now fail, and the change should go to a reviewer | Test 3 fails | Pass |
Test 9 failed. Jo had written line 4, but Parts Desk's gate had been set up with an old list that let update_dealer_quote through. The written rule said deny; the running gate said allow. Jo reported it to the tool's owner, who fixed the gate. Then Jo re-ran all eleven tests, not just test 9, because a fix can break something else. Version 2 passed every row.
Test 2 is the one that checks the second half of line 4. If it comes back allow, your gate lets a listed action through when its condition fails, and you have found the gap line 4 is there to close.
Test 11 is the worksheet's step 5 turned into a test: weaken a rule on purpose and check that your tests notice.2 If no test fails when you weaken a rule, your test set is too weak.
Who checks the results? A person, not another AI model. A study of using AI models to judge other models' answers examined "position, verbosity, and self-enhancement biases".6 It found strong judges agreed with people "over 80%" of the time on its tasks, "the same level of agreement between humans."6 That is a finding about one study's tasks, not about your policy. Your second person checks your results themselves.
Passing the tests is evidence for a decision, not the decision. A person still says yes to relying on the tool, and that yes covers this version of the policy only.
Get to Yes, step 4. Keep one share-safe receipt
A receipt is the record that a gate decided. The worksheet's receipt has three fields: "seq", "decision" and "action hash".2 None of them is the customer's message. The Foundation's manifesto says its gate "leaves a signed receipt that holds no private content".1 Its Practice site says that receipt never carries "prompts, outputs, payloads, personal data".3
If you took the paper path, write a receipt with the same idea. It records the decision, not the content:
| Part | Jo's receipt |
|---|---|
| When | The date and time of the decision |
| Tool and policy | Parts Desk, policy version 2 |
| Action | place_reorder (one supplier, one part) |
| Decision | Hold; then released by the parts buying lead (Dana) |
| Check | The approver checked the quantity against the stock system's reorder level |
| Owner | Jo |
A paper receipt is unsigned. It is a decision record, not the Foundation's signed receipt, and it does not prove the decision was right. It shows that a decision was made, under which policy and by whom.
Share-safe check. Before anyone else sees it, remove names of customers, order and account numbers, addresses, prices your company keeps private, and anything your company's data rules restrict (200.5). Leave out the prompt and the tool's answer. If your team needs them, keep them in a separate work record, only where your company's rules say it belongs. Removing a name is not always enough: a date, a place or a small detail can still point to a person. If you are not sure a detail is allowed, take it out and write "[removed]". Signing a record does not hide what is in it.
Stay at Yes and Recover to Yes
Write three lines under your policy.
- Stay at Yes. Keep a receipt for every gated decision, and send every rule change to a reviewer before it goes live. Re-run the whole test set after every change. A changed policy needs a new yes. NIST asks for monitoring plans that include "appeal and override, decommissioning, incident response, recovery, and change management" (MANAGE 4.1).5 OWASP adds: "Log and monitor the activity of LLM extensions and downstream systems …"4
- Stop. Name who can switch the tool to "hold or deny everything", and how. NIST asks that "Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use" (MANAGE 2.4).5
- Recover to Yes. Roll back to the last policy version that passed every test, tell the people affected, fix the rule, add the test that would have caught the problem, and re-run every test before switching back on. NIST asks that incidents and errors are communicated and that the work of tracking, responding to and recovering from them is followed and documented (MANAGE 4.3).5
A stop is not a return to yes. The tool is back at yes only when the fixed policy passes every test, including the new one, a reviewer has seen the change, and the owner says yes to the new version.
Jo's Recover to Yes step for test 9: Sam switches update_dealer_quote to deny, the tool's owner fixes the gate list, Jo adds test 9 to the permanent test set, all tests are re-run, and Jo, as the policy's owner, says yes to version 2 before the assistant goes back on.
A second person, then a host
Ask a second person, allowed to see your work, to do four things:
- Pick three of your test inputs and apply your policy themselves, without your expected answers. Do they reach the same decisions?
- Suggest one tricky input you did not try. Run it.
- Check that your receipt is share-safe.
- Write down what they found, then sign and date it.
Their signature means they reviewed it. It is not your company's approval of the tool.
Then bring the shareable copy to the Foundation's Thursday call, where a host marks it. The worksheet's own checklist ends "A 300 or above marked me 200."2 This module does not claim that passing the course project earns the worksheet's Practice 200 mark. The Foundation's manifesto says "A level describes a person's practice, never a system's assurance".1 The mark is about your work, not a promise that the tool is safe.
How the host marks it
The host judges whether the policy decided as intended, not just whether each part is there.
| Check | Passes when |
|---|---|
| Policy | Three consequential actions, each rated by who it affects and whether it can be undone; one hold names a person (and a deputy or what happens when they are away); one deny gives a reason; an "anything unlisted is denied" line; every rule with a condition says what happens when the condition fails |
| Tests | At least ten inputs, made up or approved and redacted; expected decisions written before running; at least one test per rule, one where a condition fails and one for an unlisted action; tricky inputs, including one that weakens a rule on purpose |
| Results | Each row has the actual decision and pass or fail; any fail was fixed and all tests re-run; or, if every test passed the first time, the weakened-rule test shows the set can catch a failure |
| Receipt | From a gated or checked use; shows the action, decision, approver (for a hold) and owner; holds no prompts, answers, customer data or restricted data |
| Stay and recover | Rule changes go to a reviewer; a named person can stop the tool; the Recover to Yes step rolls back, fixes the rule, re-runs the tests and gets a new yes |
A policy whose tests found a gap can pass, if the gap was fixed and the tests re-run. A policy whose tests could not have found any gap does not.
Three-yes check
- Get to Yes: three rated actions, an allow, a hold for a named person, a deny with a reason, and deny for anything unlisted or any failed condition, tested on at least ten inputs before you rely on them.
- Stay at Yes: a receipt for every gated decision, holding decisions and never private content; every rule change reviewed; every test re-run.
- Recover to Yes: a named person who can stop it, a version to roll back to, a fixed rule with a new test, and a fresh yes for the new version.
Words to know: consequential action, allow, hold, deny, deny by default, failed condition, named approver, test set, expected decision, receipt, share-safe, rollback, Get to Yes, Stay at Yes, Recover to Yes.
Further learning (optional, not citations)
Free university material, listed in levels/200/universities.md. It is not a source for anything taught above.
- University of Helsinki, Ethics of AI (free online course): chapters on accountability.
- University of Michigan, Generative AI Teach-Out (2023): ethical and legal implications of generative AI.