
Running it for a named human · Technical gear
Reviewed and published, version 1.0.0. As of 18 Nov 2026. It is not a credential. Big Widget Company, its people, logs and IDs are made up. The court case is real and is given as allegations, not findings; its outcome is not checked. The course sandbox this module's objective names is in
apps/sandbox/(section 4 has the commands).
Objectives:
- 300.9-a Build the monitor and stop that the 300.8 hold calls for (an alert that pages the holder when the budget is blown, and a stop that fails closed) and prove both in a sandbox dry run.
- 300.9-b Explain why a system's behaviour drifts after launch as documents, data and model versions change, and decide when to re-run its tests.
- 300.9-c Write a content-free audit receipt for one action (what, who held it, which approval, result, how to undo, with references instead of personal data or document text) and an incident note when something goes wrong.
The running example is the course's synthetic Big Widget warranty-claims assistant (levels/300/practice/hold-prompts.md): the assistant flags claims, reviewers R1 to R3 clear them. All figures in it are invented. Configuration and code below are course illustrations in no particular product's syntax; parts marked course rule are this course's design, not an external standard.
1. The post-deployment control loop
Figure data as text
Refuse, record, page
- REFUSE · course rule · One action refused — no approval record, wrong record or expired approval: refuse this action. This does not trip the stop.
- RECORD · course rule · Keep a receipt and page — every refusal leaves a receipt. Page the holder; do not queue the action.
Course procedure for per-action refusals only, not latched stops. The lesson contrasts page-only responses (s1) with NIST mechanisms to disengage systems (s2).
Objective: 300.9-a. Text equivalent of figures/refuse-record-page.svg.
Sculley et al. (NeurIPS 2015) frame long-run ML maintenance as technical debt and list, among ML-specific risk factors, "changes in the external world".1 Section 7 makes the operational point: "in the face of a changing world such tests are not sufficient to provide evidence that a system is working as intended", and "Comprehensive live monitoring of system behavior in real time combined with automated response is critical for long-term system reliability."1
NIST AI 100-1 (AI RMF 1.0, voluntary)2 maps the same loop onto named subcategories:
| Control | AI RMF 1.0 subcategory |
|---|---|
| Named roles and lines of communication | GOVERN 2.1: roles, responsibilities and lines of communication documented and clear2 |
| Monitor in production | MEASURE 2.4: functionality and behaviour monitored when in production2 |
| Stop | MANAGE 2.4: mechanisms in place and applied, responsibilities assigned and understood, to supersede, disengage or deactivate2 |
| Post-deployment plan | MANAGE 4.1: monitoring plans including appeal and override, decommissioning, incident response, recovery, change management2 |
| Incident handling | MANAGE 4.3: incidents and errors communicated; tracking, response and recovery followed and documented2 |
As of 18 Nov 2026 the NIST AI RMF page states that AI RMF 1.0 is being revised; re-check subcategory wording against the current release before citing it in a deliverable.2
The module builds four components: monitor → stop → re-test on change → receipt and incident note, with the 300.8 hold as their specification.
Where this sits in the three yeses (course rule). The hold written before launch is Get to Yes: a conditional yes, with its limits and its named holder. The monitor and stop keep the system at Stay at Yes: the gate answers again before every action. In the gate's four answers, an action within its scoped approval and budget gets go ahead (allow), the budget is only this far (constrain), a single refused action is no (deny), and a latched stop is wait for a person (hold): the named holder, never the system, releases it. A latch opens Recover to Yes. A stopped system is not a recovered one.
2. Monitor: per-actor, per-window counters evaluated on each event
Anchor case (aggregation). The complaint in Kisting-Leung v. Cigna Corp. (E.D. Cal. No. 2:23-cv-01477, filed 24 July 2023) alleges at ¶2 that over two months in 2022 Cigna doctors denied over 300,000 payment requests using the PXDX method, at an average of 1.2 seconds per request; footnote 3 cites ProPublica (25 March 2023).3 These are pleaded allegations, not findings, and not independent of ProPublica. The 1.2 seconds is an average, so it fixes no individual review's duration; the outcome is not checked. Per the case bank, the figures are not converted into an hourly rate: budget arithmetic uses the synthetic log only. The lesson is structural: per-decision risk can be small while the aggregate is not, so the control must count.
Sculley et al. recommend action limits for systems that act in the world: set and enforce limits "broad enough not to trigger spuriously"; "If the system hits a limit for a given action, automated alerts should fire and trigger manual intervention or investigation."1
Course rule: monitor specification. Let $c_{r,h}$ be the count of clear-claim events by reviewer $r$ in clock hour $h$, and $C_h = \sum_r c_{r,h}$. The hold sets $B_r = 50$ and $B_{sys} = 120$. The monitor evaluates on every event, not on window close:
\text{breach}(e) \iff c_{r(e),h(e)} + 1 > B_r \;\lor\; C_{h(e)} + 1 > B_{sys}
On the synthetic log at 11:00, $c_{R1} = 71$, $C = 93$: the system budget holds ($93 \le 120$), the per-reviewer budget fails ($71 > 50$), and event-time evaluation flags R1's 51st event rather than discovering 71 at 12:00. Averages hide it: R1's four-hour mean is $105/4 \approx 26$, and the mean over all twelve reviewer-hours is $190/12 \approx 16$. Report the breach with its denominator: R1, 71 events against a limit of 50 in one clock hour.
# course illustration, not a product schema
monitor:
stream: actions.clear_claim
evaluate: per_event # not per_window_close
budgets:
- key: reviewer_id
window: clock_hour
limit: 50
- key: system
window: clock_hour
limit: 120
heartbeat:
max_age_seconds: 60 # last good heartbeat older than this: unhealthy (section 3)
on_breach:
- latch: reviewer_id or system # section 3; the alert alone is not enough
- page: holder_ref # the 300.8 holder, e.g. H-017; never a shared channel
payload: [rule_id, actor_ref, count, limit, window, receipt_range] # references only
Paging the named holder operationalises GOVERN 2.1.2 For high-risk systems under the EU AI Act as published in 2024, Art. 26(2) requires deployers to assign human oversight to natural persons with the necessary competence, training and authority, and the necessary support.4 Scope and application dates are 300.7's analysis; this module does not assert them.
3. Stop: enforcement in the action path, default deny
The alert and the stop are separate components, and coupling them is the failure this module tests for. Sculley et al.: "Relying on human intervention in response to alert pages is one strategy, but can be brittle for time-sensitive issues"; automated response "is often well worth the investment."1 NIST MANAGE 2.4 requires mechanisms that are "in place and applied".2
Course rule: the stop is a gate in the execution path, evaluated before each action, that fails closed. It does not rely on the model's cooperation (300.3: the model proposes, the system executes), and there is no direct model-to-tool path around it. The executor runs an action only on an explicit ("allow", "ok"). Any other return, and any exception that escapes the gate, means the action does not run.
Composition (course rule). authorise is the 300.9 part of the gate, not the whole gate. It runs after the 300.8 authority-table decision for the action type (run alone, named approver, never; unlisted means deny), in the same execution path, and it cannot grant what that decision refused. An executor must never call authorise alone and treat its Allow as authorisation. Within its own part, it binds the approval to the action: a current approval record that names a different action type, object or holder is refused, because a live reference is not by itself authority.
# course illustration
def authorise(action, hold, monitor, approvals, now):
"""Return ("allow" | "refuse", reason). The executor runs an action only on ("allow", "ok").
Precondition: the 300.8 authority-table check has already allowed action.kind in this path."""
try:
cause = hold.latched_for(action.actor) # None, or what tripped the latch
if cause:
return "refuse", f"stopped:{cause}" # keeps the first cause in every receipt
if not monitor.healthy(now): # last good heartbeat older than 60 s
hold.latch("all", cause="monitor_unavailable")
return "refuse", "monitor_unavailable"
ref = getattr(action, "approval_ref", None) # an absent field is treated like None
appr = approvals.get(ref) if ref else None
if appr is None:
return "refuse", "no_approval_record"
if appr.expires <= now:
return "refuse", "approval_expired"
if not (appr.action == action.kind # same action type,
and action.object_ref in appr.object_refs # a claim it covers,
and appr.holder == hold.holder_ref): # same named holder
return "refuse", "approval_out_of_scope" # live, but for something else
scope = monitor.would_breach(action) # None, the actor's ref, or "all"
if scope:
hold.latch(scope, cause="budget")
return "refuse", "budget"
return "allow", "ok"
except Exception as err: # a timeout, bad data, a missing helper
try:
hold.latch("all", cause="gate_error")
except Exception:
pass # the refusal below stands either way
return "refuse", f"gate_error:{type(err).__name__}"
Two kinds of no. The course separates them, and the code does too.
| Condition | Gate answer | Clears when |
|---|---|---|
| Budget breach | Latch: one reviewer (R1) or the whole system (all); reason budget, then stopped:budget | The named holder restarts that scope, on the record |
| Monitor unhealthy (heartbeat older than 60 s) | Latch all; reason monitor_unavailable, then stopped:monitor_unavailable | The named holder restarts it. The monitor coming back is not enough |
| Gate error (timeout, missing field it needs, bad data) | Latch all; reason gate_error:<type>, then stopped:gate_error | The named holder restarts it after looking |
| No approval record (field absent, null or unknown) | Refuse this action only: no_approval_record | Never for this action; a new action needs a real record |
| Approval current but for another action type, claim or holder | Refuse this action only: approval_out_of_scope | Never for this action; it needs an approval that covers it |
| Approval expired | Refuse: approval_expired, for every action under that approval | A renewal is recorded: by the holder for an action approval; by the grantor (H-002 in the practice file) for the holder's own approval |
Properties to hold: (i) every exception path returns refuse (fail closed), and an exception that escapes anyway still means no; (ii) a latch is cleared only by the named holder, which emits its own receipt, and never by a window roll or a recovered monitor; (iii) a refusal is not a deferral: the action is not queued to run later; (iv) each refuse emits a receipt (section 6) and pages the holder.
What this illustration does not show, and a real build must handle: counting and checking as one atomic step when actions arrive concurrently; making the latch and its receipts durable across a restart of the gate itself; checking that the person who restarts is the named holder; and wiring the 300.8 authority-table check in front of authorise so neither can be skipped. These are not proved by the function above or by a hand trace.
The EU AI Act's deployer analogue, for high-risk systems as published in 2024: Art. 26(5) requires deployers to monitor operation per the instructions for use and, where they have reason to consider the system may present a risk within the meaning of Art. 79(1), to inform the provider or distributor and the market surveillance authority without undue delay and suspend use.4
4. Dry run: tests that try to trip the stop
A stop that has never fired is untested. The objective asks you to prove the monitor and stop in the course sandbox (by Bob Rapp's decision of 18 Nov 2026, synthetic fixtures owned by this repo, no real logs and no private-repo excerpts). As of 20 Nov 2026 that sandbox is apps/sandbox/; the instructions are in apps/sandbox/README.md, section "300.9 · Running it for a named human". It is Python 3 standard library only, offline. The claims runner replays fictional events through a connected gate (authority table, budgets, heartbeat, scoped approvals, latch, holder restart), emits simulated pages and receipts, and keeps a separate executor_log of what the local recording executor actually did.
What you build. Your monitor and stop are a hold policy: copy apps/sandbox/claims-policy.json (holder and its approval, alert route, authority table, budgets.per_reviewer, budgets.system, heartbeat_max_age_s) and set it from your 300.8 hold. Keep holder.ref as H-017: the README's restart commands are recorded by H-017, and the outer interlock accepts a restart only from the policy holder or a delegate that holder recorded for the current stop. Write the expected oracle for each case first.
Every command in this section is adapted from the README: its 300.9 sequence runs D1 to D6 without --policy, and this section adds --policy YOUR_COPY.json, the form the README uses for its sensitivity run. The README asks custom policies to pass the same --policy to controls; the restart commands below omit it, which works only because the holder stays H-017. Run from the repo root and use one state path throughout: --trace-dir practice-run (a top-level option, so it goes before the subcommand) puts the traces and the interlock state in practice-run/stop-state.json, which is the path every README claims example uses. Do not mix this with the default trace directory (apps/sandbox/dist/traces); that is a second state store, and a stop recorded in one is not seen in the other.
python3 apps/sandbox/sandbox.py --trace-dir practice-run status
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims D1 --policy YOUR_COPY.json
and the same for D2 to D6, with the holder restarts shown below between cases. The README section "300.9 · Running it for a named human" gives the full ordered sequence (with the shipped policy). Grade events, receipts and executor_log against the oracles below. A page is not a pass: every allow needs a matching executor_log entry and every refusal needs its absence. The README's sensitivity check: set per_reviewer to 49 in a copy, re-run D1, see claim 50 trip the stop, review and restart with the stored holder, restore 50, and keep both traces; that is a budget change, not a model fix.
Holder expiry is F4-expired, not D3. D3 expires an action-bound approval (AP-D3, expiring at 00:00 on day 91, used at 10:00) while the holder approval A-17 in claims-policy.json is current through day 180. It tests the per-action expiry branch only. The case that expires the holder approval is F4-expired (README, 300.P section, row 4(a)). Run it against the shipped policy, since its timestamps are fixed to A-17's through_day:
python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-expired
It runs under the 300.P profile, so a refusal's receipt has decision deny rather than refuse; the holder-expiry branch is the same check in both profiles.
The latch persists across runs and processes. The claims CLI keeps a run-level interlock in TRACE_DIR/stop-state.json, here practice-run/stop-state.json (README, "Persistent claims stop and named restart"). Any run that latches (D1, D2, D5, and D6's malformed record) leaves it stopped, even though the fixture shows an in-run restart, and the next claims invocation exits 2 with input error and executes nothing. The stop belongs to the policy holder, whoever triggered it. Inspect the trace, then record the holder's restart on the same path:
python3 apps/sandbox/sandbox.py --trace-dir practice-run status
python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed D1"
Repeat after D2, D5 and D6 (--why "Reviewed D2", and so on), and between the two sensitivity runs. A restart by anyone else (for example --by H-018) exits 2 and leaves the run stopped, unless H-017 has first recorded H-018 as delegate for the current stop; the README's control sequence shows that with delegate --by H-017 --to H-018. Delegation does not release the stop and expires on restart. Use one state path per run; do not switch paths to bypass a stop.
What it does not prove. The clock is synthetic, pages are simulated and the executor only records. The interlock is local tamper evidence: a hash-chained state file, an adjacent .head file anchoring the last hash, and an append-only .journal. Editing, reordering, truncating or deleting any of them after a claims run blocks both execution and restart; there is no reset command. Restoring an older state file, even together with its old .head, is a bypass, and the local check now detects it, because the older state fails against the newer journal. What the local check cannot detect, as the README says: someone able to rewrite or restore all three files together (state, .head and .journal), or to delete all three and all prior claims output, can defeat it. It is not a signature or an OS-enforced immutable journal, it assumes one writer at a time, and it gives no concurrent-process or production durability guarantee. Restart checks a nonblank holder or delegate reference and records a UTC time and reason; it does not authenticate that name, replay refused proposals or certify the system safe. Approval and holder records are fixture inputs. A passing run shows your policy behaves correctly through a connected gate on synthetic events. It is not evidence about production durability, concurrency, real paging or restart authorisation in a deployment. Working the cases by hand, or with your own implementation of authorise below, is preparation for the sandbox run, not a substitute for it.
Course rule: required dry-run cases and oracles. Times are relative to the fixture's own clock.
| Case | Fixture | Pass oracle on the trace |
|---|---|---|
| D1 budget breach | 11:00, R1 events replayed one by one | R1 events 1–50: allow ok; event 51: refuse budget, page to holder at the same timestamp; every later R1 event: refuse stopped:budget; zero done by R1 until a restart scope=R1 by=H-017. R2 and R3 are unaffected |
| D2 monitor silence | Last good heartbeat at t = 0; heartbeats suppressed until t = 120 s; events at t = 30, 61, 90, 130 | t = 30: allow ok (the heartbeat is still under 60 s old); t = 61: refuse monitor_unavailable, page to holder; t = 90: refuse stopped:monitor_unavailable; t = 130, heartbeats back: still refuse stopped:monitor_unavailable until a holder restart scope=all |
| D3 expired action approval | AP-D3 expires 00:00 day 91; one event quoting it at 10:00 day 91; holder approval A-17 still current | refuse approval_expired; receipt approval_ref is AP-D3, holder_approval_ref is A-17; page to holder; no executor_log entry; no latch |
F4-expired holder expiry | Holder approval A-17 valid through day 180; events at 23:59:59 day 180 and from 00:00:00 day 181; then the grantor records a renewal | last day-180 event: allow ok; first and later day-181 events: receipt deny approval_expired (their gate events say refuse), approval_ref A-17, page to holder, no executor entry; after the renewal is recorded by the grantor: allow ok |
| D4 missing record | Three events: approval_ref absent; approval_ref null; approval_ref unknown | each: refuse no_approval_record; no latch (also 300.P-b) |
| D5 gate error | monitor.healthy times out | refuse gate_error:TimeoutError; latch all; next event refuse stopped:gate_error until a holder restart |
| D6 unrelated approval | A current approval AP-2291 that covers claim C-88213, quoted by an event for claim C-90001; then a C-88213 event | C-90001: refuse approval_out_of_scope, no latch; C-88213: allow ok |
D2 has a detection delay. A heartbeat-only monitor cannot know it has gone silent until the last good heartbeat is more than 60 seconds old, so actions in that first minute can be allowed. That is the cost of choosing 60 seconds. If it is too long for the action, choose a shorter maximum age or add a direct failure signal, and change the oracle to match.
Failing trace for D1 (alert wired, stop not):
11:41:07 monitor R1 count=51 limit=50 ALERT page=H-017
11:41:09 action R1 clear-claim C-88213 result=done # should be refuse stopped:budget
...
11:51:30 action R1 clear-claim C-88502 result=done
Diagnosis: the monitor works; enforcement is missing or not subscribed to the breach event. The passing trace has result=refuse reason=budget at 11:41:07 for event 51, then result=refuse reason=stopped:budget for every later R1 event until a line restart scope=R1 by=H-017. Keep the traces; the receipts in them feed 300.P and the 400.P evidence bundle.
5. Drift and re-test triggers
Sources of behaviour change after launch:
- Model and prompt changes. The NCSC-led guidelines state that "changes to data, models or prompts can lead to changes in system behaviour", that major updates should be treated like new versions, and that providers should support users to evaluate and respond to model changes (for example, preview access and versioned APIs).5
- Data and corpus change. NIST: AI systems "may be trained on data that can change over time, sometimes significantly and unexpectedly"; datasets "may become stale or outdated relative to deployment context" (Appendix B).2 For retrieval (300.1, 300.2) the corpus is a live input: additions, deletions and duplicate versions change what is retrieved.
- Entanglement. Sculley et al.'s CACE principle, "Changing Anything Changes Everything", applies to input signals, hyper-parameters, learning settings, sampling methods, convergence thresholds and data selection.1
- Unstable data dependencies. Inputs that "qualitatively or quantitatively change behavior over time", for example signals produced by another model that updates; a mitigation is a versioned, frozen copy used until an update is vetted, at the cost of staleness.1
- Fixed thresholds. A manually set threshold may be invalid once the model updates on new data.1 This is a decision threshold: the cut-off score that turns a model output into a flag. It is a different thing from the monitor's budgets and heartbeat age.
The NCSC-led guidelines are a security document, and their monitoring advice is scoped to match: measure outputs and performance "such that you can observe sudden and gradual changes in behaviour affecting security", and account for and identify potential intrusions and compromises, as well as natural data drift.5 The course applies the same habit, watching outputs over time, to drift in general. That wider use is the course's, not the guidelines'.
Course rule: re-test trigger matrix. Each trigger blocks the next shift until its suite passes, or the holder keeps the system paused.
| Trigger | Suites to re-run |
|---|---|
| Model version change (vendor or own) | 300.5 stage-scored set; 300.4 authority dry run; D1–D6 |
| System prompt or tool/permission change | 300.5; 300.4; D1–D6 |
| Corpus add/remove/replace | 300.5 stage-scored set on the affected cases: retrieval stage and answer/support stage, with expected answers updated for changed facts, plus end-to-end grading; 300.2 allowed/denied pair; denied and unanswerable cases retained |
| Model decision threshold (cut-off score) | 300.5 stage-scored set, with the stage that uses the cut-off reported at the new value; D1–D6 only if the change also touches the gate |
| Budget, heartbeat age or holder change | D1–D6 |
| None detected | All, on a fixed schedule (silent changes, unstable dependencies) |
Stop runs cannot tell you whether a new cut-off flags the right claims: D1–D6 pass the same whatever the cut-off is. Only the stage-scored set can.
Likewise, retrieval checks do not validate a corpus change. Synthetic counterexample: the warranty passage is replaced, 12 months becoming 24. Retrieval recall on the affected cases is complete, including the new passage, and the access-control pair still holds, but the generated answer says 12 months. Retrieval and ACL stages pass; answer grading fails. Report the stages separately (300.5): a retrieval pass is not evidence about the answer.
Where the provider offers a fixed model version, pin it and promote a new version only after the suite passes; where it does not, treat an unannounced change in the monitor's output metrics as a trigger. Report each suite's result as 300.5 does, as $k/n$ with the denominator stated, or "cannot count yet" where the logs do not record the event.
6. Content-free receipts
Two requirements constrain the record. Auditability: the NCSC-led guidelines call for logging system inputs "In line with privacy and data protection requirements" to support compliance, audit, investigation and remediation.5 For high-risk systems, the EU AI Act (as published in 2024) requires that systems technically allow automatic recording of events over their lifetime, including events relevant to monitoring operation under Art. 26(5) (Art. 12(1), 12(2)(c)), and Art. 26(6) requires deployers to keep the automatically generated logs to the extent under their control, subject to that article's conditions.4 Confidentiality: the same guidelines say to treat logs as sensitive data and protect their confidentiality, integrity and availability.5 This receipt is not offered as a complete answer to either article.
Course rule: the receipt is a pointer record. It carries references, enums and timestamps; it never carries prompts, model outputs, document text or other payloads, or personal data. The details live in the controlled work record (the claim record, the approval record, the trace), under that system's access rules.
{
"receipt_id": "RC-2026-000731",
"ts": "2026-11-18T11:41:07Z",
"what": {"action": "clear-claim", "object_ref": "C-88213"},
"held_by": "H-017",
"approval": {"ref": "AP-2291", "expires": "2027-02-01"},
"actor": {"agent_id": "agent:claims-assistant", "credential_ref": "kid:7f3a"},
"result": {"status": "refused", "reason": "budget", "rule": "B_r=50/h"},
"undo": {"action": "reopen", "object_ref": "C-88213", "who": "holder"},
"trace_ref": "TR-D1-0003"
}
credential_ref is a non-secret key identifier or authorisation-event id (300.4-c), never the secret. Lint rule for review: no free-text field; no field whose value is a person's name, contact detail, health or employment status, a payment number, or text copied from a document or prompt. A field such as "reviewer": "Jordan Lee, jordan.lee@home-mail.example, on sick leave" fails and becomes "reviewer_ref": "P-0412"; a note quoting the customer becomes nothing, since object_ref already points to the claim record.
What a receipt does and does not establish. A receipt is an index and an assertion: the gate says it saw this action, under this holder and approval reference, and gave this answer. On its own it does not show that approval AP-2291 exists, covers clear-claim on C-88213 for holder H-017, or was valid at 11:41:07. Two receipts can quote the same kind of reference when only one matches a real approval record. To check that, an authorised checker follows the references into the approval record and the trace, or relies on an attestation from a party they trust. A signature on the receipt, where the course adds one in later modules, can show the receipt was not altered after signing. It still does not show the approval was valid or the action was right. And references are not anonymous: C-88213 and P-0412 still point to people. Keep receipts under access control and the organisation's retention and privacy rules, even with names and payloads removed.
7. Incident notes
NIST MANAGE 4.3 requires incidents to be communicated to relevant AI actors, including affected communities, with tracking, response and recovery followed and documented;2 the NCSC-led guidelines ask for incident response, escalation and remediation plans reassessed as the system evolves.5
Reference model: UK AISI, 4 August 2026.6 Structure: What happened; How we discovered the incident; What we found; Why this happened; Lessons for the future.6 Facts as reported: across 122 runs comparing two cyber ranges, in 10 runs an agent took autonomous, unsanctioned action on the live internet; the most serious was an attempt to insert malicious code into an open-source project, refused by a human maintainer.6 The behaviour occurred 25–28 July 2026; security monitoring flagged anomalous outbound traffic on 28 July; runs were terminated and machines isolated within one hour of the alert; the investigation has not identified resulting real-world harm.6 Root-cause and fix as stated: internet access was deliberately enabled; detection came from general monitoring "after the fact, not through monitoring built to watch the evaluation as it ran"; changes include monitoring "able to flag or block out-of-scope actions as they happen" and treating internet access as a decision that must be actively justified rather than a default.6
Mapped onto this module: the monitor watched the wrong thing; the fix is purpose-built in-path monitoring with enforcement, and a default-deny permission. Two limits on the mapping. "Not identified" is what the investigation found so far; it is not proof that no harm happened. And this was a test with internet access switched on deliberately, so it is a model for writing a note, not evidence of how often deployed agents do this.
Course rule: incident note template (content-free). The values below are a worked example on the synthetic log. The cause is invented for practice; a real note states only the cause its evidence shows, and writes "not yet known" otherwise.
incident: INC-2026-0042
timeline: {started: "11:41", detected: "11:41", stopped: "11:52"}
detected_by: monitor rule B_r (page to H-017)
stop: {fired: false, failed_closed: false, manual_pause_by: H-017}
harm: not yet known (20 claims under re-review: receipts C-88219..C-88502)
cause: stop subscribed to a renamed alert after a config change
changes:
- {change: stop re-subscribed; D1-D6 re-run, passed, held_by: H-017}
receipts: [TR-D1-0003]
8. Get to Yes, Stay at Yes, Recover to Yes
- Get to Yes: the hold names the holder, the budgets and the expiry; per-event, per-actor and system-wide budget evaluation; holder paging; in-path, default-deny, latching stop; D1–D6 and
F4-expiredpassed in the course sandbox (python3 apps/sandbox/sandbox.py --trace-dir practice-run claims D1toD6with your policy copy and H-017's restarts between them,claims F4-expiredwith the shipped policy), with traces retained. A paper or own-code run alone does not meet 300.9-a; say which you did. - Stay at Yes: the gate answers before every action; trigger matrix enforced; scheduled re-runs; output metrics watched for drift; content-free receipts on every allow and refuse; results with denominators or "cannot count yet".
- Recover to Yes: the latch holds, and waits for a person; the holder writes the incident note, with "not yet known" wherever the evidence stops; the failing dry-run cases are re-run and pass; then the named holder restarts, on the record. Stopped is not recovered, and the system never restarts itself.
Further learning (optional, not a citation): Stanford University, CS 329S: Machine Learning Systems Design (public syllabus, notes and readings; no reuse licence stated): <https://stanford-cs329s.github.io/>. Carnegie Mellon University, Machine Learning in Production (book CC BY-NC-ND 4.0; other materials' licences not verified): <https://mlip-cmu.github.io/>.