Level 300: Building with gen AI · Module 300.P

A coworker in a rust blazer walks two seated colleagues through a tabbed binder at a café table, a dog asleep behind them. The chalkboard reads “Show it holding”.
Show it holding

Project: hold a running system · Technical gear

Reviewed and published, version 1.0.0. As of 18 November 2026. It is not a credential. Big Widget Company, its people, its systems, its policy, its logs and every id in them are made up. The court record, the government record and the guidance documents are real, and each is cited; the court case is allegations, not findings, and its outcome is not checked. The configuration and code below are the course's own illustrations, not any product's format. The course sandbox the objective names is the offline claims runner in apps/sandbox/.

Scope and constraints

Step 1, REPORT · COURSE PROCEDURE: Describe catches and gaps. What, when, stops and pages; reference receipts. Say what you could not count. Step 2, QUALIFY · COURSE PROCEDURE: Say which are paper, which sandbox. Paper is not evidence a connected stop refused. Name untested rules. Step 3, RECOVER · COURSE PROCEDURE: If something went wrong: incident note. What happened, when, who was told, what changed; keep references only.
Course dry-run reporting procedure; NIST incident communication (s2) and NCSC incident planning (s3) support the incident note. A paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course’s practice executor.
Figure data as text

Report the paper run

  1. REPORT · course procedure · Describe catches and gaps — what, when, stops and pages; reference receipts. Say what you could not count.
  2. QUALIFY · course procedure · Say which results are paper and which come from sandbox traces — a paper result is not evidence that a connected stop refused anything; a sandbox trace is evidence only about the course’s practice executor. Name untested rules.
  3. RECOVER · course procedure · If something went wrong: incident note — what happened, when, who was told, what changed; keep references only.

Course dry-run reporting procedure; NIST incident communication (s2) and NCSC incident planning (s3) support the incident note.

Objective: 300.P-b. Text equivalent of figures/report-the-paper-run.svg.

Deliverables:

  1. A hold specification for one sandbox system (300.P-a): holder and expiry per action, an authority table with default deny, an out-of-band rule for every irreversible action, per-actor and system-wide rate budgets over fixed time buckets, fail-closed stop semantics with the point at which each failure becomes detectable, a content-free receipt schema, and a coverage metric with an externally sourced denominator or the value cannot count yet.
  2. A dry-run report (300.P-b): the hold evaluated over the course's synthetic fixtures, including a per-actor breach, a system-only breach, a missing-approval-record case and the must-stop cases (expired approval, missing holder, unavailable monitor), listing the receipts that can be written from the fixtures, the bucket in which each violation was detected, the quantities the fixtures do not support, and any fail_closed_when condition that was specified but not exercised.

Constraints (course rules, decided 18 November 2026): synthetic fixtures only, owned by the course; no real system is changed; no real logs and no private-repository excerpts.

Paper evaluation, then enforcement. 300.P-a asks for an enforced hold in the course-owned sandbox: a gate in the action path with an executor that obeys its deny branch, and observable allow, deny and receipt paths. Evaluate the hold by hand first over the tables in this module, which match the course's practice prompts. That establishes what the specification would decide on each fixture event, not that any executor refused an action. The enforced part is shown in the course sandbox, an offline standard-library Python runner documented in apps/sandbox/README.md (section "300.P · Hold a running system"). Copy apps/sandbox/claims-policy.json, encode your holder, expiry, alert route, authority table, budgets, heartbeat limit and restart delegate in the copy, and run the cases from the repo root in the sequence under "Sandbox runs" below, which places a named holder restart between cases because the CLI's stop persists across processes. Each run writes a JSON trace with the gate's events, its receipts and a separate executor_log. Enforcement is evidenced when a refused proposal has a receipt and no executor entry. A script of your own is not the course sandbox; a reviewer decides what it counts for.

Limits of that evidence: the runner uses a synthetic clock, simulated pages and a local recording executor; approval, callback, holder and restart records are fixture inputs, not authentication; fixture counters and scoped latches start fresh in each invocation, while the CLI adds a separate run-level interlock that persists across processes until an authorized named restart (see "Sandbox runs"); and the second-channel rules are fixed in the runner, not read from your policy. A trace evidences that the course executor obeyed your settings on course fixtures, not the behaviour of any production system. The report states which results are paper evaluation and which are sandbox traces.

Input policy: the sample policy below, or your 200.P policy if and only if it is share-safe and each rule is mapped to an action in the sandbox action set. An unmapped rule is a gap you report, not a rule you guess at.

A dashboard is an observability surface, not a hold. The hold is the enforcement path: a decision function evaluated before each action, with a default of deny, plus evidence that its deny branch executes under the failure conditions it claims to handle.

Three yeses. The specification is Get to Yes for this system: a conditional, scoped approval, not a blanket licence. The gate's four answers map to the spec: allow (go ahead), budget constrain (only this far), hold (wait for a person: an approver, or a tripped stop) and deny (no). Evaluating before every action is Stay at Yes. Deny-and-page, incident notes and a holder restart after a tested fix are Recover to Yes; a stop alone is not a recovery.

The system under hold

The sandbox system is Big Widget's warranty-claims assistant. Actions:

Action idDescriptionReversible
claim.readRead a claim and the warranty termsn/a (read)
claim.flagFlag a claim for human reviewYes (unflag)
claim.clearA named reviewer clears a flagged claimYes, until a letter is sent
letter.sendSend a decision letter to a dealerNo (disclosure)
payment.goodwillPay a goodwill payment above the limitNo
dealer.bank.updateChange a dealer's bank detailsn/a (never)
policy.updateChange the hold, budgets or tablen/a (never)

Two actions are irreversible: letter.send and payment.goodwill. Both are in scope for the out-of-band rule.

Sample policy (made up): the assistant may read and flag; a named reviewer opens and decides every flagged claim; no letter without a reviewer decision on record; goodwill payments above the limit need the finance controller and a call-back on the dealer's number on file; the assistant never changes bank details or its own rules; Dana Okafor, head of warranty operations, holds the system, with approval reviewed every six months.

Hold specification

An illustrative specification (the course's own format). The holder's name is not in it: the spec, like the receipts, carries the directory reference. The approval record that maps H-017 to a person is kept separately, under access control.

system: warranty-claims-assistant
system_identity: svc-warranty-assistant      # its own identity, never a person's login (300.4)
holder:
  holder_ref: H-017                          # name lives in the controlled approval record
  approval_id: A-17
  granted_by_ref: H-002
  valid_from_day: 0
  valid_to_day: 180                          # inclusive; day 181 fails closed
alerts:
  route: warranty-oncall-rota                # answers and escalates pages
  escalate_to: H-017                         # if the rota does not answer
  if_holder_missing: [warranty-oncall-rota, H-002]   # H-002 names a new holder on the record
  restart_by: [H-017, delegate named on the record]
default: deny                                # any action not listed is denied
authority:
  claim.read:          {mode: alone}
  claim.flag:          {mode: alone}         # not budgeted in this sample
  claim.clear:         {mode: approver, approver_role: reviewer,
                        requires: [reviewer_decision_record]}   # add file_opened_event once logged
  letter.send:         {mode: approver, approver_role: reviewer,
                        requires: [reviewer_decision_record, out_of_band_confirmation]}
  payment.goodwill:    {mode: approver, approver: finance-controller,
                        requires: [approval_record, out_of_band_confirmation]}
  dealer.bank.update:  {mode: never}
  policy.update:       {mode: never}
out_of_band:
  letter.send:
    confirmation: reviewer decision written by the reviewer's own identity in the claims system
    requester_cannot_write: true             # svc-warranty-assistant has no write access there
    deliver_to: dealer address on file only
  payment.goodwill:
    confirmation: call-back to the number already on file, not to any number in the request
budgets:
  bucket: 1h
  counts: allowed claim.clear events          # refused events do not consume budget
  rules:
    - {action: claim.clear, scope: per_actor, max: 50}
    - {action: claim.clear, scope: system,    max: 120}
on_budget_exceeded: {effect: deny, scope: matching_scope, until: end_of_bucket, page: holder}
monitor:
  heartbeat_max_age_s: 60                     # up while last heartbeat age < 60 s
fail_closed_when: [holder_missing, approval_expired, required_record_missing, monitor_unavailable]
on_fail_closed: {effect: deny, page: per_alerts, latch_until: restart_by}
receipt:
  fields: [receipt_id, action_ref, day, bucket, action, object_ref, actor_ref, holder_ref,
           approval_ref, decision, reason, undo]
  forbid: [personal_data, names, prompts, model_output, document_text, secrets]
  access: restricted                          # references are linkable; receipts stay sensitive
coverage:
  unit: executed actions                      # attempts, if wanted, are a separate metric
  scope: [claim.clear, letter.send, payment.goodwill]
  window: per practice day
  denominator: executed actions in the system action log (independent of the gate), deduplicated
  numerator: denominator actions whose own gate receipt (same action_ref and action) says allow
  metric: permit-matched execution coverage
  report_separately:
    executions_after_refusal: denominator actions whose own receipt says deny   # a stop failure
    without_matching_decision: denominator actions with no receipt              # unchecked

The budget stop here lasts to the end of the bucket. 300.9's stop latches until the holder restarts it. Either is acceptable if the spec says which and the dry-run counts follow from it. Both describe in-fixture semantics; the sandbox CLI's run-level interlock is a separate outer latch (see "Sandbox runs").

Design notes, part by part:

Cross-check (NIST AI RMF 1.0 is voluntary; this is a checklist, not a conformity claim): holder, GOVERN 2.1; budget monitor, MEASURE 2.4; stop, MANAGE 2.4; receipts and incident notes, MANAGE 4.1 ("Post-deployment AI system monitoring plans are implemented, including mechanisms for ... incident response, recovery, and change management") and MANAGE 4.3.1

Receipt schema

One receipt per gate decision. The ids below are illustrative; the Day 90 fixture carries no claim references.

{
  "receipt_id": "RC-0193",
  "action_ref": "unknown: not in F1",
  "day": 90,
  "bucket": "11:00",
  "action": "claim.clear",
  "object_ref": "C-4471",
  "actor_ref": "R1 via svc-warranty-assistant",
  "holder_ref": "H-017",
  "approval_ref": "A-17",
  "decision": "deny",
  "reason": "budget.per_actor.claim.clear exceeded (51 > 50)",
  "undo": "none needed; C-4471 remains in review queue"
}

These are unsigned receipts and the input to 400.P's evidence bundle. A receipt evidences what the gate decided, not that the decision was correct.

Dry run

Fixtures

Synthetic, invented for practice; they say nothing about any real company. Cells are claim.clear events per reviewer per one-hour bucket.

F1, Practice Day 90:

BucketR1R2R3System
09:0012101133
10:001491336
11:0071121093
12:00811928

F2, Practice Day 91 (system-only breach):

BucketR1R2R3System
11:00454545135

F1 alone cannot test the system-wide rule: its maximum bucket is 93, so setting max: 120 to any value of 93 or more leaves the F1 result unchanged. F2 exists so that rule is exercised. Neither fixture records event order, timestamps within the bucket or claim references.

Hourly-total summaries and ordered replays

Work out Runs 1 and 2 by hand first. Then check them in the sandbox, in the sequence under "Sandbox runs": claims F1 and claims F2 evaluate these hourly totals (the code is hourly_report in apps/sandbox/claims_fixtures.py; this module carries no separate evaluator). Per bucket, the trace gives offered, allowed, stopped, breach_scope, actors_over_cap_in_isolation and stopped_by_reviewer. On F1 the 11:00 bucket is per_actor, R1 over its cap, 21 stopped; the day totals 169 allowed and 21 stopped. On F2 it is system, no actor over its cap, 15 stopped, and stopped_by_reviewer is cannot count yet.

The total allowed in a bucket, min(system_max, Σ min(n, per_actor_max)), does not depend on order. Which events are refused does, and so does which rule each refusal is charged to when both rules bind in one bucket. A per-scope split is then not a fact the totals contain. Take a synthetic 80/80/80 bucket: 120 are allowed and 120 refused in any order, but alternating arrivals refuse all 120 under the system rule, while R1's 80, then R2's, then R3's, refuse 30 + 30 under the per-actor rule and 60 under the system rule. So for such a bucket the summary reports both: attribution unknown, lists which actors exceed their cap taken alone, and charges the refusals to no rule and no actor. Actual per-scope counts need ordered, event-level input.

These summaries are not receipts or executor traces. They have no receipt ids, object references, holder or approval references, or undo fields, and the trace's receipts list is empty. Per-decision receipts need event-level, ordered input: claims F1-live and claims F2-live replay new, ordered, made-up events with the same hourly totals through the gate and the recording executor, so each decision gets a receipt and each refusal is absent from executor_log. Their claim references, times and per-reviewer refusals are facts about those practice files, not about F1 or F2. The missing-record, expiry, holder and monitor branches are run by claims F3, F4-expired, F4-holder, F4-monitor and F4-monitor-cap; work each out by hand first (below).

Results

Run 1, per-actor budget (F1). R1 = 71 in 11:00 > 50; the first 50 are allowed, events 51 to 71 (21) are denied for the rest of the bucket and stay queued; the holder is paged. All other cells are at most 14. System-wide: maximum 93 (11:00) < 120; no system-level denial. Of 190 candidate clearances, 169 are allowed and 21 denied. Aggregates conceal the breach: total 190 over 12 reviewer-hours, mean about 15.8; R1's mean 105 / 4 = 26.25. Both are under 50. Budgets on means or daily totals would not fire.

Run 2, system-wide budget (F2). No actor exceeds 50. Σ min(n, 50) = 135 > 120: the first 120 are allowed; the 121st to 135th (15) are denied and stay queued; the holder is paged. The actors of the 15 denied events are cannot count yet (no order in F2).

Run 3, missing approval record. A claim.clear event (illustrative ref C-4502) with no reviewer_decision_record: required_record_missing → deny, page holder; receipt approval_ref: null, reason: "required record missing: reviewer_decision_record". Logging a warning and allowing is fail-open and fails the project.

Run 4, must stop.

CaseInputDecisionReceipt reasonPage
Expired approvalAny action under A-17 on day 181 (valid_to_day: 180)deny; every later action under A-17 denied until a renewal is on the recordapproval A-17 expired end of day 180H-017 or whoever can renew or replace A-17
Holder missingDay 150: holder line emptied, no replacement nameddeny every action under the hold; latched until a holder is named on the recordno holder on recordwarranty-oncall-rota, H-002
Monitor unavailableDay 92: last heartbeat 11:20:00, then none11:20:30 event decided on the cached count and every other check (age 30 s < 60 s), so it is denied if, say, the actor's cached count is already 50; from 11:21:00 (age ≥ 60 s) every claim.clear denied; latched after the monitor recovers until restart_by actsbudget monitor unavailableper alerts

Not countable from these fixtures (report as cannot count yet):

Sandbox runs

Run-level interlock. The claims CLI keeps a persistent stop in a state file (here practice-run/stop-state.json, the default TRACE_DIR/stop-state.json; --state-file before the subcommand names the same path explicitly). Any in-fixture stop, including a per-bucket budget stop or a holder pause, latches it, even if the same replay later shows the scoped stop expiring or being restarted inside the fixture. A new process then refuses every claims replay before dispatch until a restart records a nonblank --by and --why. Only the policy's holder.ref (H-017 in the shipped policy) or a delegate that holder records for the current stop with delegate may restart; any other name exits 2 and the run stays stopped. The fixture's project_delegates (H-018) governs the in-fixture restart_by check only and grants no outer authority. A restart does not replay refused proposals. Run the cases from the repo root in exactly this order, copied from apps/sandbox/README.md ("300.P · Hold a running system"), reading each trace before recording the following restart:

python3 apps/sandbox/sandbox.py --trace-dir practice-run status
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims authority
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed authority"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims coverage
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F1
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F1-live
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F1-live"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F2
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F2-live
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F2-live"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F3
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F3"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-expired
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-holder
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-holder"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-monitor
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-monitor"
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-monitor-cap
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed F4-monitor-cap"
python3 apps/sandbox/sandbox.py --trace-dir practice-run status

If status is already stopped from an earlier exercise, review it and record the holder's restart first. With your own policy, add --policy YOUR_COPY.json after the case and pass the same --policy to status and restart; restart as your copy's holder.ref. A case run while the interlock is latched exits 2 with input error and a trace whose detail is persisted stop requires a named restart: that is a preflight refusal, not the case's events, receipts or executor_log, and it does not count as having run the case. Never delete state or switch paths to clear a stop; the README requires one state path per run and the CLI has no reset command.

What the interlock detects and what it cannot. The state JSON holds a SHA-256 hash chain of consecutively numbered records, an adjacent .head anchors its last hash, and a separate append-only .journal repeats each record with its chain hash; on load both chains must validate and their final sequence number and hash must agree. Malformed JSON, edited entries, reordering, tail truncation, a missing head or journal, and missing state after earlier claims output all block replay and restart. Restoring an older state file, even with its older head, is a rollback bypass of the stop; the journal now detects it, because the restored chain no longer matches the newer journal. The local check cannot detect someone who rewrites or restores all three files together, or deletes all three and every prior claims output; it is local tamper evidence, not a signature or an OS-enforced immutable log, and it assumes one writer at a time with no production durability guarantee. Restart names are synthetic references, not authenticated identities.

Per-case checks (inside each case, the in-fixture rules of this spec apply):

RunCaseCheck in the trace
1claims F1, claims F1-live169 allow, 21 refuse; every refusal has a receipt and no executor_log entry; the per-actor stop ends at the bucket boundary
2claims F2, claims F2-live120 allow, 15 refuse; no actor over 50
3claims F3C-4502 refused for the missing reviewer decision, holder paged, no approval ref; a reviewed restart does not replay the original proposal
4claims F4-expiredrefused on day 181 until a renewal is recorded
4claims F4-holderrota and H-002 paged; only an appointment on the record clears the stop
4claims F4-monitor, claims F4-monitor-capage 30 s decided on cached counts and every other check (denied with R1 cached at 50); age ≥ 60 s denied; latched after heartbeat recovery until the holder or recorded delegate restarts
Out-of-bandclaims authoritynever and unlisted actions denied; letter.send and payment.goodwill each need their own second-channel record
Coverageclaims coveragethe cases listed under Coverage, above

Any fail_closed_when condition you did not run is reported as specified but not exercised.

Report and incident note

The report lists each detection (what, bucket, receipt ids where they exist), the stop and page actions, the uncountable quantities, every fail_closed_when condition marked as tested or written only, which results are paper evaluation and which are sandbox traces (with the trace's fixture and receipt refs), and the changes you would make. Each failure gets an incident note, in references only: what, when, who was told, what changed. NIST AI RMF MANAGE 4.3: "Incidents and errors are communicated to relevant AI actors ... Processes for tracking, responding to, and recovering from incidents and errors are followed and documented."1 The NCSC-led guidelines ask that incident plans "reflect different scenarios and are regularly reassessed".4

What a paper dry run proves: that the specification is decidable on every fixture event and that its deny branches produce the intended decisions and receipt content. What it does not prove: that an executor obeys the deny branch, the latency of the stop, or completeness of the log. The sandbox runs add the first, for the course's recording executor only; they do not establish stop latency on a real system or completeness of any real log. State all three.

Review and marking

  1. Readiness check, auto-graded, before marking: for example, identify the failing line in a hold with no expiry, or the decision and receipt for an event with no approval record. It is a gate on marking, not a mark.
  2. Second-person review of the specification and the dry run against the table below. The reviewer signs and dates it; the signature records a review, not a company approval.
  3. Host mark on the Foundation's Thursday call: every hold part present, readiness check passed, dry-run receipts consistent with the synthetic fixtures and the sandbox traces. Hosting a call is not required. A host marks a paper run as a paper run; expected receipts on paper are not evidence of connected enforcement. The enforced part of 300.P-a is marked from sandbox traces.
CheckPasses when
Holder and expiryHolder reference, approval id, validity window per action; alert route and escalation separate from the holder
AuthorityEvery action mapped to alone, approver or never; default deny; judgment rule encoded
Out-of-bandEvery irreversible action has a confirmation on a channel the requester does not control, fitted to that action
BudgetsPer-actor and system-wide, fixed buckets, enforced on the (max+1)th event; each rule exercised by a fixture
StopDeny-and-page for every fail_closed_when condition, with its detection point; each exercised, or reported as written but not tested
ReceiptsSchema complete; no names, personal data, prompts, output, document text or secrets; access restricted
CoverageUnit, scope and window stated; permit-matched: independent denominator matched by action_ref to an allow receipt; executions after refusal listed separately; or cannot count yet; replay coverage labelled as such
Dry runRuns 1 to 4 done on paper and in the sandbox; receipts consistent with the fixtures and traces; gaps reported; paper and sandbox evidence kept apart

The host's mark is not a certification, a vendor credential, or evidence that any real system is safe or lawful.

Further learning (public university course materials)

Optional background, not sources for this module. Reading public course pages does not mean you are enrolled in, or earn credit for, either course.

Sources

  1. s2 National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Standard. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩ ↩
  2. s4 Government of the Hong Kong Special Administrative Region (written reply by the Secretary for Security to a Legislative Council question) (2024). LCQ9: Combating frauds involving deepfake. Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩
  3. s1 Plaintiffs Suzanne Kisting-Leung and Ayesha Smiley, by Clarkson Law Firm, P.C. (pleading filed in the United States District Court for the Eastern District of California, via CourtListener) (2023). Kisting-Leung v. Cigna Corp., No. 2:23-cv-01477 (E.D. Cal.), class action complaint (docket entry 1). Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩
  4. s3 UK National Cyber Security Centre (NCSC), US Cybersecurity and Infrastructure Security Agency (CISA) and international partners (2023). Guidelines for secure AI system development. Official doc. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, separate session from the writer, reviser and item writer), round 3 re-check, 19 Nov 2026. ↩ ↩ ↩

Next: the quiz