Level 300: Building with gen AI · Module 300.P

A coworker in a rust blazer walks two seated colleagues through a tabbed binder at a café table, a dog asleep behind them. The chalkboard reads “Show it holding”.
Show it holding

Project: hold a running system · Plain gear

Reviewed and published, version 1.0.0. As of 18 November 2026. It is not a credential. Big Widget Company, its people, its systems, its rules and its logs are made up. The court record, the government record and the guidance documents are real. The court case is a set of claims, not findings. How it ended is not checked.

Your project

Step 1, REPORT · COURSE PROCEDURE: Describe catches and gaps. What, when, stops and pages; reference receipts. Say what you could not count. Step 2, QUALIFY · COURSE PROCEDURE: Say which are paper, which sandbox. Paper is not evidence a connected stop refused. Name untested rules. Step 3, RECOVER · COURSE PROCEDURE: If something went wrong: incident note. What happened, when, who was told, what changed; keep references only.
Course dry-run reporting procedure; NIST incident communication (s2) and NCSC incident planning (s3) support the incident note. A paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course’s practice executor.
Figure data as text

Report the paper run

  1. REPORT · course procedure · Describe catches and gaps — what, when, stops and pages; reference receipts. Say what you could not count.
  2. QUALIFY · course procedure · Say which results are paper and which come from sandbox traces — a paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course’s practice executor. Name untested rules.
  3. RECOVER · course procedure · If something went wrong: incident note — what happened, when, who was told, what changed; keep references only.

Course dry-run reporting procedure; NIST incident communication (s2) and NCSC incident planning (s3) support the incident note.

Objective: 300.P-b. Text equivalent of figures/report-the-paper-run.svg.

This is the last project in Level 300. You take one practice AI system and you hold it.

Holding a system means writing down, for every job it does: who answers for it, what it may do, what makes it stop, and what record it leaves. Then you test what your rules would do.

You bring two things:

  1. Your hold. The written rules for one practice system (300.P-a).
  2. Your test report. What your rules would have done on a practice log (300.P-b).

Everything is pretend. You never touch a real system or real records. The course gives you a made-up system, made-up rules and made-up logs.

Test on paper first, then on the practice computer. First you do the test by hand, with the logs in this lesson. A paper test shows what your rules would do. It cannot show that a computer did stop. The project asks for a stop that a computer actually obeys. So next you run the same tests in the course's practice computer setup, called the sandbox. It is a small program in this course's files. The steps are in apps/sandbox/README.md, in the part called "300.P · Hold a running system".

The sandbox is still pretend. It sends no real letters, payments, calls or messages. When it stops, it stays stopped until the holder starts it again (see "Now run it in the sandbox" below). It only shows that the course's practice computer obeyed your rules. In your report, say which results came from paper and which came from the sandbox.

Your starting rules. Use the sample rules below. If you did the 200.P project, you can use your own rules instead. But only if nothing in them is private, and only if you match each rule to the practice system's jobs. If a rule does not match any job, say so. Do not guess. If you can't do both, use the sample rules.

People make every choice here. You write the hold. A second person checks it. A host marks it.

The three yeses. In 100.9 you met Get to Yes, Stay at Yes and Recover to Yes. Your hold is the Get to Yes. It says what may run and who answers for it. Before every job, the hold gives one of four answers: go ahead, only this far, wait for a person, or no. Answering every time is Stay at Yes. Stopping, paging a person and writing a note start Recover to Yes. A stopped system is not fixed yet.

Here is a common mistake: "If a system has a dashboard, someone is holding it." A dashboard only shows numbers. A hold says who answers, what may run, and what stops it. And someone has seen the stop work on a running system.

The practice system

Big Widget Company makes widgets. Dealers send in warranty claims when a widget breaks. An AI helper reads the claims. It marks, or flags, the ones it thinks the warranty does not cover.

Three reviewers, called R1, R2 and R3, look at the flagged claims. When a reviewer signs off one flagged claim, we call that a clearance.

The sample rules (made up):

The AI helper may read claims and the warranty rules, and flag claims. A named reviewer must open each flagged claim and decide it. No letter goes to a dealer unless a reviewer decided. Big goodwill payments need the finance boss to say yes, and a phone call back to the dealer's number on file. The helper never changes a dealer's bank details. It never changes its own rules. Dana Okafor runs warranty work and holds the helper. Her yes is checked every six months.

Your job is to turn these rules into a hold.

The seven parts of a hold

1. One named person, and an end date

One real person holds each job. "The team" is not enough. A team can share the work. A team rota can answer an alarm at 3 a.m. and pass it up. But one named person must answer for why the job was allowed to run. The rota answers the alarm. The holder answers for the yes.

Each yes also has an end date, because people move jobs.

A US science agency, NIST, wrote a guide to AI risk. It says who does what should be written down and clear to everyone.1

Example: Dana Okafor holds the helper. Her code on receipts is H-017. Her yes is called A-17. It ends after Day 180. Alarms go to the warranty team rota first. If nobody answers, they go to Dana. If there is ever no holder at all, the alarm also goes to the boss who gave Dana her yes (code H-002). That boss must name a new holder. (The practice files count days, not real dates.)

2. The "who may do what" table

Every job goes in one of three boxes: alone, needs a named person to say yes, or never. Anything not on the list is a no.

JobBoxThe hold's answer
Read a claimAloneGo ahead
Flag a claimAloneGo ahead
Clear a flagged claimA named reviewer, who opens the claim and decidesWait for a person, then only this far
Send a letter to a dealerA named reviewer, whose choice is on record, plus a second checkWait for a person
Pay a big goodwill paymentThe finance boss, plus a call backWait for a person
Change a dealer's bank detailsNeverNo
Change its own rulesNeverNo
Anything elseNeverNo

The AI helps. A person decides. A person who signs without looking has not decided.

The limit in part 4 counts clearances only. Flags have no limit in these rules. If you want one, give flags their own limit and their own count.

3. A second check for things that cannot be undone

Some acts cannot be undone. This system has two. Sending money is one. Sending a letter is the other: once a dealer reads it, you cannot take it back. For both, the check must come through a second channel that the one asking does not control.

Here is why. In 2024, the Hong Kong Government reported a fraud the Police got in late January 2024. Crooks used deepfakes, which are fake videos made with AI. They made a fake video meeting from video clips found online.2 The person they tricked said yes to sending money to five local bank accounts. The loss was about HK$200 million.2 The meeting was recorded ahead of time. Nobody real was talking back.2 The government did not name the company, and neither do we. In general advice, not about this case, the Police told people to check with a phone call when a relative or friend asks for money by video or audio.2

So a face on a video call does not prove who is asking.

For the helper:

4. A limit on how many

Small acts can add up to something big. So set a limit for each hour: one for each reviewer, and one for the whole system. Look at the busiest hour, not the average.

Here is a warning. In 2023, people sued an insurance company called Cigna. Their court papers say its doctors turned down over 300,000 payment requests in two months in 2022. They say the doctors spent an average of about 1.2 seconds on each.3 They also say the doctors did not open the patient files.3 These are claims in a lawsuit, not findings by a court. How the case ended is not checked. We use them as a warning, not as numbers to do maths with.

Example limit: each reviewer may clear up to 50 an hour. The whole system may clear up to 120 an hour. Go over, and clearances stop for the rest of that hour, and Dana gets a page.

5. A stop that fails closed

"Fails closed" means: if something is missing, the answer is no. If the holder, the yes, a record or the alarm is missing, the job waits. A person gets a page.

Some problems show up at once, like a missing record. Others take a while. The alarm sends a "still here" signal. In these rules, if the last signal is a minute old, the alarm counts as gone. So for up to a minute, the hold does not know yet. Write that down.

NIST's guide says there should be ways to switch an AI system off when it is not working as planned, with someone in charge of doing it.1 A stop only counts once you have seen it stop something on a running system.

For the helper, "stopped" means the claim waits for a person. Nothing is turned down or paid by itself.

6. A receipt with no private details

Every time the hold decides something, it leaves a receipt. The receipt says: what happened, who held it, which yes, what the result was, and how to undo it.

Use codes, never names or the claim's words. That goes for Dana too. The receipt says H-017, not "Dana Okafor". Her name stays in the approval record, which only allowed people can see. The UK's cyber-security agency says to treat logs as sensitive and protect them.4 A receipt with only codes lets someone check the choice without copying private details.

Codes are not secret names. A person who is allowed to can look them up. Codes also do not hide everything. Someone who already knows a claim's code from somewhere else can still match it to a receipt. And the same code on many receipts links them together. So codes keep names and words out of the receipt, but the receipts are still sensitive. Keep them where only allowed people can read them, and do not post them anywhere.

Example receipt (made up): RC-0193. Clear claim C-4471. Reviewer R1. Holder H-017. Yes A-17. Result: stopped, R1 is over 50 this hour, holder paged. Undo: nothing to undo, the claim is still waiting.

These receipts go up to Level 400. There, you pack them so a stranger can check them. A receipt shows what the hold decided. It does not prove the choice was right.

7. An honest count

Say how many of the system's jobs your hold saw, out of how many. The "out of how many" must come from some other record, not from your hold's own receipts. If you don't have one, write "cannot count yet". That is an honest answer.

Count the same kind of thing on both sides. Say first what you are counting: jobs that actually ran, or jobs that were asked for. Then match them one by one. Take each job in the other record and ask: did the hold leave a receipt for this exact job, and does that receipt say yes? Only those jobs count. A receipt for a job the hold said no to does not count for a different job that ran. Just dividing one total by the other can look like 100% when the hold said yes to none of the jobs that ran.

Watch for one more case. A job ran, and its own receipt says no. That job does not count. It is worse than a missing receipt: the stop did not work. List it on its own line in your report.

The test run

Now you test your hold on the practice logs. Nothing real happens. You just write down what your hold would have done. A paper test does prove something. It shows whether your rules give a clear answer each time, and what the stops and receipts should look like. It does not prove that a live system obeys them. Say that in your report.

Test every problem your hold says it handles. If you wrote a rule but did not test it, say "written, not tested".

The practice log

These numbers are made up. Each box is how many claims a reviewer cleared in that hour, on Day 90.

HourR1R2R3Whole system
9:0012101133
10:001491336
11:0071121093
12:00811928

The log only has hourly totals. It has no claim codes, no minutes, and no order.

Test 1: each reviewer's limit

What the hold does: R1's first 50 clearances that hour are fine. At number 51 the hold stops R1 for the rest of the hour. The other 21 claims wait. Dana gets a page. So of 190 clearances, 169 go ahead and 21 wait.

When did it catch it? In the 11:00 hour. The log only has hourly totals, so you cannot tell the exact minute. Say so.

Test 2: the whole-system limit

Test 1 never gets near 120. So it does not test the whole-system limit at all. Here is a second made-up hour, on Day 91:

HourR1R2R3Whole system
11:00454545135

No reviewer gets to 50. But the whole system cleared 135, over 120. The hold lets the first 120 through. It stops the other 15. Dana gets a page. Whose clearances were stopped? The log does not show who came in what order. Write "cannot count yet".

Test 3: a missing record

One clearance has no record of the reviewer's choice. The hold must not let it through. It stops it and pages Dana. The receipt says: "Missing: no reviewer choice on record."

If the hold just writes a warning and lets it through, it failed open. That is a fail.

Test 4: it must stop

Each of these must stop. The receipt says why. Someone who can fix it gets a page. Nothing carries on by itself.

Now run it in the sandbox

Write down what you expect first. Then run the tests from the top folder of the course files.

The sandbox remembers a stop. When a test makes the sandbox stop, it stays stopped. Closing it and starting it again does not clear the stop. The stop is saved in a file, practice-run/stop-state.json. Only the holder, H-017, can start it again, by typing her code and a reason. (She can also name someone on the record to do it for her.) Nobody else can. So between tests, you play Dana: you look at what the last test did, then you record the restart. Never delete the file or switch to a new folder to get around a stop. That is a way around the stop, not a fix.

Type these lines in this order. They are copied from apps/sandbox/README.md, part "300.P · Hold a running system":

python3 apps/sandbox/sandbox.py --trace-dir practice-run status
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims authority
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed authority"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims coverage
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F1
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F1-live
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F1-live"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F2
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F2-live
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F2-live"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F3
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F3"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-expired
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-holder
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-holder"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-monitor
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-monitor"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-monitor-cap
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-monitor-cap"
python3 apps/sandbox/sandbox.py --trace-dir practice-run status

status tells you if the sandbox is ready or stopped. If it is still stopped from an earlier exercise, look at why, and record the holder's restart first.

Which line goes with which test:

TestCase names
1F1, then F1-live
2F2, then F2-live
3F3
4F4-expired, F4-holder, F4-monitor, F4-monitor-cap
Second check (part 3)authority
Honest count (part 7)coverage

Each run tells you where it saved its record. Open the record. A stop worked if the job has a "no" receipt and is missing from the list of jobs that ran (executor_log).

If you skip a restart, the next test does not run at all. It prints "input error", and its record says "persisted stop requires a named restart". That record is not a test result. Record the restart and run that test again.

To try your own rules, copy apps/sandbox/claims-policy.json and change your copy. Add --policy and your copy's name after the case name. Add the same thing to status and restart too. If you change the holder's code in your copy, restart with that code.

Two kinds of stop. Inside one test, the stops follow the rules in this lesson. A budget stop ends when the hour ends, and R2 and R3 carry on. The sandbox's saved stop is a second, outside lock. It holds the whole sandbox until the holder restarts it, even when the stop inside the test has already ended. It does not change the test's numbers: Test 1 is still 169 go ahead and 21 wait.

The sandbox is honest about its limits. It does not check who really typed "H-017"; it only checks the code. It notices if someone edits, cuts short, deletes or swaps in an old copy of the saved stop file. But someone who replaces all of its files together, or deletes all of them and every past record, can still fool it. It is a practice lock, not a real one.

F1 and F2 only add up the hourly totals. The -live runs use new made-up jobs with the same totals, one by one. Their codes and times belong to those practice runs, not to the Day 90 log.

What you could not count

End with the gaps. For these logs:

If something goes wrong, write a short note: what happened, when, who you told, and what you changed. Use codes, not names. NIST says problems should be shared with the right people and written down.1 The UK guide says to plan for different kinds of problems and to check the plans again and again.4 The system is back at yes only when the fix is tested and the holder says so.

Checking and marking

  1. A quick quiz first. The computer marks it. For example: a hold has no end date; which line fails? You pass this before anyone marks your project. The quiz is not your mark.
  2. A second person checks your hold. They look at all seven parts and your four tests. They check that no receipt has names or private details, and that you said which results are paper and which came from the sandbox. Then they sign and date it.
  3. A host marks it on the Foundation's Thursday call. The host checks that all seven parts are there, your quiz passed, and your receipts match the practice logs and the sandbox records. You do not have to run a call. The host marks a paper test as a paper test. Only a sandbox record shows that the stop was obeyed.

The host's mark means a host checked your work. It is not a certificate. It does not prove any real system is safe or legal.

Sources

  1. s2 National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Standard. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩
  2. s4 Government of the Hong Kong Special Administrative Region (written reply by the Secretary for Security to a Legislative Council question) (2024). LCQ9: Combating frauds involving deepfake. Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩
  3. s1 Plaintiffs Suzanne Kisting-Leung and Ayesha Smiley, by Clarkson Law Firm, P.C. (pleading filed in the United States District Court for the Eastern District of California, via CourtListener) (2023). Kisting-Leung v. Cigna Corp., No. 2:23-cv-01477 (E.D. Cal.), class action complaint (docket entry 1). Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩
  4. s3 UK National Cyber Security Centre (NCSC), US Cybersecurity and Infrastructure Security Agency (CISA) and international partners (2023). Guidelines for secure AI system development. Official doc. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩

Next: the quiz