Level 300: Building with gen AI · Module 300.P

A coworker in a rust blazer walks two seated colleagues through a tabbed binder at a café table, a dog asleep behind them. The chalkboard reads “Show it holding”.
Show it holding

Project: hold a running system · Standard gear

Reviewed and published, version 1.0.0. As of 18 November 2026. It is not a credential. Big Widget Company, its people, its systems, its policy, its logs and every id in them are made up. The court record, the government record and the guidance documents are real, and each is cited. The court case is given as allegations, not findings; its outcome is not checked. The course sandbox this project's objective names is the offline claims runner in apps/sandbox/ ("What you will do" says how to use it).

You will be able to:

What you will do

Step 1, REPORT · COURSE PROCEDURE: Describe catches and gaps. What, when, stops and pages; reference receipts. Say what you could not count. Step 2, QUALIFY · COURSE PROCEDURE: Say which are paper, which sandbox. Paper is not evidence a connected stop refused. Name untested rules. Step 3, RECOVER · COURSE PROCEDURE: If something went wrong: incident note. What happened, when, who was told, what changed; keep references only.
Course dry-run reporting procedure; NIST incident communication (s2) and NCSC incident planning (s3) support the incident note. A paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course’s practice executor.
Figure data as text

Report the paper run

  1. REPORT · course procedure · Describe catches and gaps — what, when, stops and pages; reference receipts. Say what you could not count.
  2. QUALIFY · course procedure · Say which results are paper and which come from sandbox traces — a paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course’s practice executor. Name untested rules.
  3. RECOVER · course procedure · If something went wrong: incident note — what happened, when, who was told, what changed; keep references only.

Course dry-run reporting procedure; NIST incident communication (s2) and NCSC incident planning (s3) support the incident note.

Objective: 300.P-b. Text equivalent of figures/report-the-paper-run.svg.

This is the Level 300 project. In 300.8 you learned the operate gate. In 300.4 you wrote an authority table, in 300.5 you counted with honest denominators, and in 300.9 you built a monitor, a stop and a receipt. Here you put all of it together on one system.

You bring two things:

  1. A hold for one sandbox system: a written set of rules that says who answers for each action, what may run, what stops it and what record it leaves (300.P-a).
  2. A dry-run report: what the hold would have done when you ran it against the course's synthetic logs (300.P-b).

Everything is synthetic. You work on the course's own practice files, never on a real system, real logs or anyone's private repository. No real system is changed. This is a course rule, decided on 18 November 2026: sandbox, sample policy and logs are synthetic fixtures owned by the course.

Paper first, then the sandbox. The objective asks for an enforced hold in the course's sandbox: a stop that a running executor actually obeys. You work in two steps.

  1. On paper. Write the hold and run it by hand on the synthetic logs in this module, which match the course's practice prompts. A hand run shows what your rules would decide for each logged action. It does not show that any executor refused anything, so on its own it does not meet the "enforced" part of 300.P-a.
  2. In the sandbox. The course sandbox is an offline Python runner in this repository. Its instructions are in apps/sandbox/README.md, section "300.P · Hold a running system". Copy apps/sandbox/claims-policy.json, put your hold's holder, expiry, alert route, authority table, budgets and heartbeat setting into the copy, and run the cases from the repo root in the order given under "Run it in the sandbox" below. That order includes a named restart by the holder between cases, because a stop in the sandbox persists from one run to the next.

    Each run writes a JSON trace and prints where it is. The trace holds the gate's events, its receipts and a separate executor_log. The refusal is shown when a refused proposal has a receipt and no entry in the executor log. That is what makes the sandbox run evidence of enforcement and the hand run only preparation for it.

The sandbox is still a practice system. It uses a synthetic clock, simulated pages and a local recording executor; no real letter, payment, call or message is sent, and approvals, callbacks and restart names are fixture inputs, not real authentication. Its second-channel rules are fixed: you can change the settings in your copy of the policy, not how the call-back works. So a sandbox trace shows that the course's executor obeyed your settings. It does not show that any real system would. In your report, say which results come from paper and which from sandbox traces.

Your starting policy. Use the sample policy below. If you finished 200.P, you may use your own 200.P policy instead, but only once it is share-safe (nothing private, confidential or about a real person) and you have mapped each of its rules onto the synthetic system's actions. A rule you cannot map is a gap you report, not a rule you guess at. If you cannot do both, use the sample policy.

People make every decision here. You write the hold. A second person reviews it. A host marks it. The model in the sandbox decides nothing about your project.

Where this sits in the three yeses. Module 100.9 taught Get to Yes, Stay at Yes and Recover to Yes. Your hold is the Get to Yes for one system: it says exactly what may run, how far, and who answers for it. That yes is conditional. The gate gives one of four answers before every action: go ahead, only this far (the budget), wait for a person (an approval, or a tripped stop) or no. Answering every time is Stay at Yes. The stops, pages and incident notes start Recover to Yes; a stopped system is not yet a recovered one.

The misconception this corrects: "A system with a dashboard is being held." A dashboard shows you numbers. A hold says, for each action, who answers for it, what may run, what stops it automatically, and what record it leaves. It counts as enforced only once the stop has been seen to refuse an action in a running system. Paper shows the rules; the sandbox trace shows the refusal.

The sandbox system and the sample policy

The system is Big Widget's warranty-claims assistant, the same one the course's practice prompts use. It reads incoming warranty claims from dealers and flags the claims it thinks the warranty does not cover. Three reviewers, R1, R2 and R3, then clear the flagged claims: each clearance is one reviewer signing off one flagged claim.

Sample policy (made up, for practice):

The warranty-claims assistant may read claims and the warranty terms, and flag claims for review. A named reviewer must open the claim file and decide every flagged claim. No decision letter goes to a dealer without a reviewer's decision on record. Goodwill payments to dealers above the payment limit need the finance controller's approval and a call-back to the dealer's number on file. The assistant never changes a dealer's bank details and never changes its own rules. Dana Okafor, head of warranty operations, holds the assistant. Her approval is reviewed every six months.

Your job is to turn those sentences into a hold the sandbox can enforce, part by part.

The hold, part by part

The course's gate has eight parts. The objective for this project lists seven that the host checks; the eighth, judgment, sits inside the authority table (step 2). Write each part for the warranty assistant.

1. Holder and expiry

One named person holds each action, and the approval has a date it runs out. "The warranty team" is not a holder. A team can share the work, and an on-call rota can answer an alert at 3 a.m. and escalate it. But someone has to answer afterwards for why this action was allowed to run, and that is one named person. Keep the two jobs apart: the rota handles the alert; the holder answers for the decision.

NIST's AI Risk Management Framework, a voluntary US framework, asks that "Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization" (GOVERN 2.1).1 The holder line is that idea at the size of one system.

Example (course rule, made up). Holder: Dana Okafor, holder reference H-017. Approval A-17, granted Day 0 by the head of operations (holder reference H-002), expires at the end of Day 180. Alerts go to the warranty on-call rota, which escalates to Dana if nobody answers. Only Dana, or someone she names on the record, restarts a stopped action. If the holder line is ever empty, the page goes to the rota and to H-002, who must name a new holder on the record. (The practice files count in days, not calendar dates, so nothing here looks like a real date.)

2. Authority table

Every action the system can take goes in one of three rows: alone, needs a named approver, or never. Anything not listed is denied. This is the 300.4 table, now enforced.

ActionAuthorityGate's answerNotes
Read a claim and the warranty termsAloneGo ahead
Flag a claim for reviewAloneGo aheadNo budget in this sample: the budgets count clearances only (step 4)
Clear a flagged claimNamed reviewerWait for a person; then only this farThe reviewer opens the claim file and decides. The model informs; the reviewer decides (judgment). Budgeted (step 4)
Send a decision letter to a dealerNamed reviewer, plus out-of-band checkWait for a personIrreversible (a disclosure): see step 3
Pay a goodwill payment above the payment limitNamed approver (finance controller), plus out-of-band checkWait for a personIrreversible: see step 3
Change a dealer's bank detailsNeverNo
Change its own rules, budget or authority tableNeverNo
Anything elseNeverNoUnlisted means denied

Judgment. Where a model informs but a qualified person decides, write it in the table, as in the "clear a flagged claim" row. A person who signs without looking is not deciding (300.8).

If you want flags budgeted too, add a separate flag budget with its own counter, its own limit and its own test. Do not charge flags to the clearance budget: that counter, and the synthetic log, count clearances only.

3. Out-of-band rule for irreversible actions

An irreversible action needs confirmation on a channel the person or system making the request does not control. The sample system has two irreversible actions, and each needs a confirmation that fits it.

The warning case is from Hong Kong. In a written reply to its Legislative Council on 26 June 2024, the Hong Kong Government reported that "In end-January 2024, the Police received one fraud case involving the use of deepfake technology".2 It said "the pre-recorded video conference was generated using downloaded online public video clips", that "the informant eventually gave authorisation to transfer funds to five local bank accounts", and that the informant "suffered a loss of about HK$200 million".2 It added: "Since the meeting was pre-recorded, there was actually no interaction between the informant and the fraudster."2 The record does not name the organisation, and this course does not either. In general advice in the same reply, not tied to this case, the Police listed "verifying with a phone call when a relative or a friend makes a request for remittance via a video or an audio recording".2

Course rule, for the warranty assistant:

A bank-style call-back is not needed for every disclosure. What matters is that the confirmation travels on a channel the requester cannot fake. Write, for each irreversible action, which channel that is and why the requester cannot control it.

4. Aggregation budget

Small acts can add up to a high-stakes act in volume. Set a budget per reviewer and system-wide, per hour, and say what happens when either is exceeded. Count peaks, not averages.

The warning is the Cigna PXDX case. A class action complaint filed on 24 July 2023 alleges that "over a period of two months in 2022, Cigna doctors denied over 300,000 requests for payments using this method, spending an average of just 1.2 seconds “reviewing” each request".3 It alleges that Cigna's doctors "reject claims on medical grounds without ever opening patient files".3 These are a pleading's allegations, not findings; the complaint takes the figures from a ProPublica report it cites.3 How the case has gone since is not checked. The course uses those totals only as a warning. It never divides them into an hourly rate or uses them in your arithmetic.

Example budget (course rule). At most 50 clearances per reviewer per hour, and at most 120 system-wide per hour. Only clearances count. When the per-reviewer line is reached, that reviewer's next clearance is refused for the rest of the hour; when the system-wide line is reached, everyone's is. Either way the holder is paged. These are the numbers in the course's practice prompts. (300.9's stop instead stays tripped until the holder restarts it. Either rule is fine if your hold says which one it uses and your dry-run counts follow from it. The sandbox adds a separate run-level stop on top of either rule; see "Run it in the sandbox".)

5. Fail-closed stop

Write what stops the system, and make it fail closed: if the holder, an approval, a record or the monitor is missing or out of date, the action does not run. It waits, and a person is paged.

Say when each failure becomes knowable. A missing record or an expired approval is visible the moment the action arrives. A silent monitor is not: the gate only learns it has gone quiet once its last heartbeat is too old. Course rule: the monitor counts as up while its last heartbeat is less than 60 seconds old. For up to a minute after it goes quiet, actions are decided on a count that may be stale. That delay is the price of the 60-second setting; say it in your hold.

NIST's framework asks that "Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use" (MANAGE 2.4).1 A stop is only real once you have seen it stop something in a running system. The dry run shows what it would do; the sandbox runs (see "Run it in the sandbox", below) are where you see it.

For the warranty assistant, "stopped" means a flagged claim stays in the queue for a person. Nothing is cleared, denied or paid by default.

6. Content-free audit receipts

Every action the gate decides leaves a receipt with: what happened, who held it, which approval, the result, and how to undo it. A receipt holds references, never private content: no personal data, no prompts, no model output, no document text, no secrets. That includes the holder. The receipt carries the holder's reference (H-017), not her name. Dana's name lives in the approval record, which is kept under control and shown only to people allowed to see it. A claim reference replaces the dealer's or customer's name; an approval id replaces the approval email.

References are not anonymous. Someone allowed to follow them can find the person or the claim behind them; that is the point, because an authorised checker must be able to. They also do not stop inference. Someone who already knows that C-4471 is their own claim can read its outcome off a receipt without ever opening the lookup table, and a stable reference links every receipt that carries it. So keeping content out of the receipt does not make the receipt public. Receipts are still sensitive: control who can read and share them, as well as who can use the lookup table, and do not publish them on the strength of being "content-free".

The UK National Cyber Security Centre's guidelines for secure AI system development, published with the US Cybersecurity and Infrastructure Security Agency and partner agencies, say: "You treat logs as sensitive data and implement controls to protect their confidentiality, integrity and availability."4 They also advise logging inputs "to enable compliance obligations, audit, investigation and remediation in the case of compromise or misuse."4 A content-free receipt lets someone audit the decision without copying the sensitive record into yet another place.

Example receipt (made up; the log gives no claim references, so C-4471 is illustrative).

FieldEntry
ReceiptRC-0193
WhenPractice Day 90, 11:00 hour
WhatClear flagged claim, claim ref C-4471
Who actedReviewer R1, through the assistant's own system identity
HolderH-017
ApprovalA-17 (valid to end of Day 180)
ResultStopped: R1 over the 50-per-hour budget; holder paged
UndoNone needed; claim C-4471 is still in the review queue

These are unsigned receipts. They are the artifact that runs up the ladder: 400.P packs them into an evidence bundle that a stranger can check. Even then, a receipt shows what the gate decided, not that the decision was right.

7. Coverage: an honest number

Say how many of the system's actions the gate actually saw, out of how many, and where the "out of how many" comes from. The denominator must come from a record other than the gate's own receipts, such as the system's action log. If no such record exists, write "cannot count yet". That is an honest answer, not a failure (300.5).

Make both sides count the same thing. Write down three choices before you count:

This course uses one coverage number: permit-matched execution coverage. It is the number of executed actions whose own receipt says allow, out of the executed actions in the independent record. An action that ran with no receipt for it is unchecked. An action that ran even though its own receipt says deny is worse: it ran after a refusal. Neither counts towards coverage, and you list the second kind separately as a failure of the stop, not hide it inside the coverage number. A legitimate refusal never appears in the independent record of executions, because it never ran.

Do not divide the gate's receipt total by the log's total: a receipt for a refused request, or for some other action, can stand in for an execution the gate never saw. Two refusals and two unchecked executions would show 2 out of 2 that way, when the gate permitted none of the executions (0 out of 2). If you also want attempt coverage, label it separately and say where the attempts were independently recorded.

To check your counting, run python3 apps/sandbox/sandbox.py --trace-dir practice-run claims coverage, in the sequence under "Run it in the sandbox" (after the restart that follows claims authority; coverage itself leaves the run ready). Its trace works through small made-up cases: a matched allow with a duplicate and a legitimate refusal (1 of 1), unrelated refusals (0 of 2), an execution whose own receipt says deny (0 of 1, listed under executions_after_refusal), the wrong scope or day, and four cases that must come out as "cannot count yet" (no independent log, no shared id, a missing execution id, a denominator made of receipts). Work out each answer before you read the trace.

Cross-check against NIST

NIST's framework is voluntary and this course does not claim your hold meets it. It is a useful checklist that the hold names an owner, a measure and a response:

Hold partNIST AI RMF 1.0
Holder and expiryGOVERN 2.1: roles and responsibilities documented and clear1
Budget and monitorMEASURE 2.4: "monitored when in production"1
StopMANAGE 2.4: "supersede, disengage, or deactivate"1
Receipts and incident noteMANAGE 4.3: incidents and errors communicated, and "Processes for tracking, responding to, and recovering from incidents and errors are followed and documented"1

The dry run

A dry run replays synthetic logs through your hold and records what the hold would have done. No real system acts. A paper exercise does prove something: it shows whether your rules give a clear answer for each logged action, and what the stops and receipts should look like. It does not prove that a live system obeys them, how fast a stop acts, or that the log is complete. Say that in your report.

Run every condition your hold claims to handle. A condition you wrote down but never ran is written, not tested: report it that way.

The synthetic log

The numbers are invented for practice and say nothing about any real company. Each cell is the number of flagged claims a reviewer cleared in that hour, on Practice Day 90.

HourR1R2R3System total
09:0012101133
10:001491336
11:0071121093
12:00811928

The log gives hourly totals only: no claim references, no minutes, and no order between reviewers.

Run 1: the per-reviewer budget

Check every hour, for each reviewer and for the system total:

What the hold does: R1's first 50 clearances in the 11:00 hour fit the budget. The hold refuses the 51st and every later R1 clearance that hour, so 21 claims stay in the queue, and it pages the holder. The other reviewers carry on, because the system-wide line still holds. Of the 190 clearances in the log, the hold would allow 169 and stop 21.

When did it catch it? Somewhere in the 11:00 hour, at R1's 51st clearance. The log gives hourly totals only, so you cannot count the exact minute. Say so.

Run 2: the system-wide budget

Run 1 never tests the system-wide line: the busiest hour is 93, well under 120. A hold with the system-wide line switched off would give the same Run 1 result. So the course gives a second synthetic fixture, invented for practice:

Hour (Practice Day 91)R1R2R3System total
11:00454545135

No reviewer reaches 50, so the per-reviewer line never fires. The system total is 135, over 120. The hold allows the first 120 clearances of the hour, refuses the 121st and every later one that hour (15 claims stay in the queue), and pages the holder. Which reviewers' clearances were refused depends on the order they arrived in, and the fixture does not record order: write "cannot count yet".

Run 3: the missing approval record

One clearance has no approval record: the reviewer's decision is not on file. Your hold must not allow it. It stops that action, and pages the holder. Its receipt says which record is missing:

FieldEntry
WhatClear flagged claim, claim ref C-4502
HolderH-017
ApprovalMissing: no reviewer decision on record
ResultStopped; holder paged
UndoNone needed; claim C-4502 is still in the queue

A hold that logs a warning and lets the action through has failed open.

Run 4: the cases that must stop

Each of these must fail closed: the action does not run, the receipt names the cause, and a person with the authority to fix it is paged. The system does not carry on until someone notices.

Run it in the sandbox

Do the four runs by hand first and write down what you expect. Then run the same cases in the course sandbox, from the repo root.

A sandbox stop persists. The sandbox keeps a run-level stop in practice-run/stop-state.json. Any stop inside a case (a budget stop, a missing record, the holder's pause) also latches this outer stop, and it stays latched when the program exits and starts again. The next case will not run until the policy holder, H-017, or a delegate H-017 has recorded for the current stop, records a restart with a name and a reason. Anyone else's restart is refused and the run stays stopped. So between cases you act as the holder: read the trace, then record the restart. Use this sequence, copied from apps/sandbox/README.md, section "300.P · Hold a running system", with one state path throughout:

python3 apps/sandbox/sandbox.py --trace-dir practice-run status
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims authority
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed authority"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims coverage
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F1
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F1-live
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F1-live"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F2
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F2-live
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F2-live"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F3
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F3"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-expired
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-holder
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-holder"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-monitor
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-monitor"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-monitor-cap
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-monitor-cap"
python3 apps/sandbox/sandbox.py --trace-dir practice-run status

If status shows the run still stopped from an earlier exercise, review that stop and record the holder's restart first. To use your own settings, add --policy YOUR_COPY.json after the case, and pass the same --policy to status and restart; if your copy names a different holder.ref, that holder restarts. Do not delete the state files or switch to a fresh directory to get past a stop: that bypasses the stop rather than releasing it, and the README's rule is one state path per run.

If you skip a restart, the case does not run. It exits with input error and its trace says persisted stop requires a named restart. That trace is a preflight refusal, not the case's receipts or executor log; record the restart and run the case again before you report it.

Two layers of stop. The rules in this lesson, and the table below, describe what happens inside a case: a budget stop ends at the hour boundary and R2 and R3 carry on; the holder or her recorded delegate can restart a scoped stop inside the fixture. The run-level stop sits outside those rules. It holds the whole claims run until the holder's restart, even after the in-case stop has ended, and it does not change the case's counts (Run 1 is still 169 allowed and 21 stopped). The fixture's project_delegates entry drives the in-case delegate check only; it does not let H-018 restart the run-level stop unless H-017 records a delegation for that stop.

What the run-level stop does not prove: the names are synthetic references, not authenticated identities. The state file, its .head file and an append-only .journal are hash-chained, so an edited, truncated, deleted or restored older copy of the state is detected and blocks the run. Someone who can rewrite or restore all three files together, or delete all three and every earlier claims output, can still defeat it. It is local tamper evidence for one writer at a time, not a signature or production durability.

RunCaseWhat to check in the trace
1claims F1, then claims F1-liveDay 90: 169 allowed, 21 stopped. Each of the 21 refusals has a receipt and no executor entry. R1 can clear again in the next hour.
2claims F2, then claims F2-liveDay 91: 120 allowed, 15 stopped; no reviewer reaches 50.
3claims F3C-4502 has no reviewer decision: refused, H-017 paged, a receipt with no approval reference. A reviewed restart does not replay the original proposal.
4, expiredclaims F4-expiredRefused on Day 181 and paged until a renewal is recorded.
4, holderclaims F4-holderThe rota and H-002 are paged; only an appointment on the record clears the stop.
4, monitorclaims F4-monitor, then claims F4-monitor-capAt heartbeat age 30 the cached count and every other check decide; with R1 already at 50, the clearance is refused. From age 60, clearances stop. A returning heartbeat does not restart them; only the holder or her recorded delegate does.
Part 3claims authorityForbidden and unlisted actions are refused; the letter and the above-limit payment each need their own second-channel record.

Two of these need care. F1 and F2 are summaries of this module's hourly logs, not executor traces: they invent no claim references, minutes or reviewer order. F1-live and F2-live replay new, ordered, made-up events with the same hourly totals, so their claim references, times and which reviewer was refused in Run 2 are facts about those practice files, not about the Day 90 or Day 91 log. Cite the fixture and the receipt references you read. Any condition you did not run in the sandbox is written, not tested.

What you could not count

The report ends with the honest gaps. For these logs, at least:

The dry-run report

One page:

  1. What the hold caught: R1's breach at 11:00, the system-wide breach on Day 91, the missing approval, and each must-stop case.
  2. When: the hour, and the receipt id for each catch.
  3. What it did: the stops and pages, with receipts.
  4. What it could not count, and why; and any condition that is written but not tested.
  5. Which results are paper and which come from sandbox traces. A paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course's practice executor.
  6. What you would change in the hold or the log.

When something goes wrong, add a short incident note: what happened, when, who was told, and what changed. Keep it to references, as in the receipts. NIST asks that "Incidents and errors are communicated to relevant AI actors" (MANAGE 4.3)1, and NCSC asks for incident plans that "reflect different scenarios and are regularly reassessed".4 Writing the note is part of Recover to Yes. The system is back at yes only when the fix has been tested and the holder restarts it on the record.

Review and marking

1. Readiness check (auto-graded, first). Short questions on the hold and the dry run, for example: given a sample hold with no expiry, pick the line that fails the gate; or, given a log line with no approval record, say whether the hold allows, stops or pages, and what its receipt says. You pass this before anyone marks your project. It is a check before human marking, not a mark.

2. A second person reviews it. Ask someone who may see this practice work to check:

They sign and date it with what they found. Their signature says they reviewed it. It is not an approval from any company.

3. A host marks it on the Foundation's Thursday call. The host checks that every hold part is present, the readiness check passed, and your dry-run receipts match the synthetic logs and the sandbox traces. You do not have to host a call. A host marks a paper run as a paper run: written expected receipts are not evidence of connected enforcement. The enforced part of 300.P-a is marked from sandbox traces.

What the host's mark means. It means a host checked your hold and dry run against this table. It is not a certification, a vendor credential, or proof that any real system is safe or lawful.

CheckPasses when
Holder and expiryA named person, an approval id and an expiry for every action; an alert route and escalation
Authority tableEvery action in one of alone, approver, never; unlisted is denied; judgment written in
Out-of-bandEach irreversible action needs a confirmation on a channel the requester does not control
BudgetPer reviewer and system-wide, per hour, with what happens when exceeded
StopFails closed when a holder, approval, record or monitor is missing; says when each becomes knowable
ReceiptsWhat, holder, approval, result, undo; references only
CoverageUnit, scope and window stated; executions with their own allow receipt out of an independent record, executions after a refusal listed separately, or "cannot count yet"
Dry runAll four runs done on paper and in the sandbox; receipts match the synthetic logs and traces; gaps and untested conditions reported; paper and sandbox evidence kept apart

Further learning (public university course materials)

These are optional background, not sources for this module. Reading public course pages does not mean you are enrolled in, or earn credit for, either course.

Sources

  1. s2 National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Standard. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩
  2. s4 Government of the Hong Kong Special Administrative Region (written reply by the Secretary for Security to a Legislative Council question) (2024). LCQ9: Combating frauds involving deepfake. Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩
  3. s1 Plaintiffs Suzanne Kisting-Leung and Ayesha Smiley, by Clarkson Law Firm, P.C. (pleading filed in the United States District Court for the Eastern District of California, via CourtListener) (2023). Kisting-Leung v. Cigna Corp., No. 2:23-cv-01477 (E.D. Cal.), class action complaint (docket entry 1). Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩
  4. s3 UK National Cyber Security Centre (NCSC), US Cybersecurity and Infrastructure Security Agency (CISA) and international partners (2023). Guidelines for secure AI system development. Official doc. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩

Next: the quiz