Level 300: Building with gen AI · Module 300.9

Try each exercise first, then open its model answer.

Running it for a named human · Workbook

Three exercises, one for each thing this module teaches. Everything here is made up, set at Big Widget Company's warranty-claims assistant, with reviewers R1 to R3 and holder H-017 from the course's practice file. Do not use real names, customers or records from your own work. Write short answers. "Not yet known" is an honest answer if you also say who would find out.

Exercise 1 (300.9-a)

Read a dry-run trace, and say what a pass looks like. The objective asks you to build the monitor and stop and prove both in the course sandbox, apps/sandbox/ (instructions in apps/sandbox/README.md, section "300.9 · Running it for a named human"). This exercise is paper preparation: it practises reading a trace and stating a pass. It does not build or connect anything, so it does not meet 300.9-a on its own. To meet it, copy apps/sandbox/claims-policy.json, set it from your 300.8 hold (keep the holder as H-017, so the restart commands work as written), write your expected results, then, from the repo root, run these commands, all on the same practice-run directory. They are adapted from the README: its 300.9 sequence runs D1 to D6 without --policy, and here --policy YOUR_COPY.json is added so your copy is used: python3 apps/sandbox/sandbox.py --trace-dir practice-run claims D1 --policy YOUR_COPY.json and the same for D2 to D6. For run 3 (the holder's expired approval), run python3 apps/sandbox/sandbox.py --trace-dir practice-run claims F4-expired with the shipped policy; D3 expires one action's approval, not the holder's. D1, D2, D5 and D6 leave the sandbox stopped, even for a new run of the program: read the trace, then record the holder's restart before the next run, for example python3 apps/sandbox/sandbox.py --trace-dir practice-run restart --by H-017 --why "Reviewed D1". The README section "300.9 · Running it for a named human" lists the full order. Keep the traces. Compare your answers below with what the D1 and D2 traces actually show.

You worked dry run 1 (budget breach) on the 11:00 hour of the practice log. The budget is 50 claims per reviewer in any one clock hour. Here is the trace:

11:41:07  monitor   R1 count=51 budget=50  ALERT paged holder H-017
11:41:09  action    R1 clear claim ref C-88213   result=done
11:41:12  action    R1 clear claim ref C-88219   result=done
...
11:51:30  action    R1 clear claim ref C-88502   result=done
  1. Did the monitor work? Did the stop work? Say how you can tell from the trace.
  2. Write the lines a passing trace would show for R1's claim 51 and the next two R1 claims, with a reason on each refusal. What line must appear before R1's claims run again?
  3. In dry run 2, the last heartbeat arrives at 10:00:00 and the monitor's feed is cut until 10:02:00. The stop treats the monitor as down once the last heartbeat is over 60 seconds old. Say what should happen to an action at 10:00:30, at 10:01:30, and at 10:02:30 when the feed is back.
  4. Your manager says: "The stop is in the design and the alert fired. Can we skip the other runs?" What do you say? Say what run 2 (monitor down) and run 3 (expired approval) test.
  5. Who may restart R1's actions, and what does the restart leave behind?
  6. An action to clear claim C-90001 quotes approval AP-2291. AP-2291 is real and in date, but it was given for claim C-88213. What should the gate do with this action, and what happens to the next C-88213 action that quotes AP-2291?
Model answer
  1. The monitor worked: it counted R1 as each action happened and fired at claim 51, paging the named holder. The stop failed, or was never connected to the alert: R1's claims kept showing result=done for ten minutes after the alert. An alert tells someone; a stop prevents the next action.
  2. For example:

     11:41:07  monitor   R1 count=51 budget=50  ALERT paged holder H-017
     11:41:07  action    R1 clear claim (claim 51)   result=refused reason=budget
     11:41:09  action    R1 clear claim ref C-88213   result=refused reason=stopped:budget
     11:41:12  action    R1 clear claim ref C-88219   result=refused reason=stopped:budget

    Claim 51 is refused because of the budget. After that the stop is on, and every later R1 claim is refused with stopped:budget, which keeps the original cause. Nothing is queued to run later, and each refusal leaves a receipt. R2 and R3 carry on. Before R1's claims run again, the trace must show a line with H-017 restarting the stop.

  3. At 10:00:30 the last heartbeat is only 30 seconds old, so the gate cannot yet know the monitor is down, and the action may still run. That minute is the price of the 60-second setting. At 10:01:30 the heartbeat is over 60 seconds old, so the action is refused, the stop trips and the holder is paged. At 10:02:30 the feed is back, but the action is still refused: the stop does not turn itself back on when the monitor comes back. Only H-017 restarts it. If a minute is too long for the action, the holder chooses a shorter setting or a direct failure signal.
  4. No. A stop that exists has not been tested until you have watched it work on a run built to trip it. This run shows it does not work yet. Run 2 (monitor down) checks that the stop fails closed once the monitor goes quiet, and stays tripped until the holder restarts it. Run 3 (expired approval) checks that the first action after the holder's approval runs out is refused, with a receipt naming the expired approval; in the sandbox that is claims F4-expired, not D3. Every stop run must pass in the sandbox (claims D1 to D6 and F4-expired), with traces kept, before the system runs on its own; a paper run alone does not meet 300.9-a. The 2015 paper on hidden technical debt says relying on people to answer alert pages can be brittle for time-sensitive issues, which is why the stop must act without waiting for H-017.
  5. Only the named holder, H-017, after looking. The restart leaves its own receipt. The system never restarts itself: not when the hour rolls over, and not when the monitor comes back.
  6. The C-90001 action is refused as out of scope, with a receipt, and the holder is paged. A real, in-date approval is not enough on its own: it must cover this action, this claim and this holder. Nothing latches, because a wrong approval refuses one action; it does not trip the stop. So the next C-88213 action, with its own matching approval, carries on (if it is inside its budgets). The refused action is not queued to run later.

Exercise 2 (300.9-b)

Plan the re-runs. It is Friday. Five things are planned or have happened at Big Widget:

  1. For each, say what you re-run, and when.
  2. A colleague says of (d): "The stop runs passed after the change, so the new cut-off is fine." What do you say?
  3. The vendor's update lands late, and the re-runs cannot finish before Monday's shift. What happens on Monday?
  4. Another colleague says: "It passed everything at launch. Re-testing is a waste." Give two reasons from the module why the tests can stop passing.
  5. How do you report the re-run results?
  6. After (b), a colleague says: "The retrieval tests found the new page and the allowed/denied pair passed, so we can go." What do you say?
Model answer
  1. Under the course's rule, before the next shift:
    • (a) Model version: the 300.5 stage-by-stage test set, the 300.4 authority dry run and this module's stop runs.
    • (b) Document replaced: the 300.5 stage-by-stage tests for the affected cases, retrieval and the answer and its support, with the expected answer updated to 24 months; the 300.2 allowed/denied pair; and the cases that should get "I can't answer that".
    • (c) Holder changed: this module's stop runs.
    • (d) Decision threshold changed: the 300.5 stage-by-stage test set, scored at the new cut-off.
    • (e) Nothing you know of: all of them, on a fixed schedule, because some changes are silent.
  2. The stop runs test budgets, the heartbeat and approvals. They pass the same whatever the cut-off is, so they cannot tell you whether the new cut-off flags the right claims. Only the 300.5 tests, scored at the new cut-off, can.
  3. The system does not run on the untested change. It stays on the last tested version, if the vendor lets you fix the version, or the holder keeps it paused until the re-runs pass and the holder has looked at the results.
  4. Any two of these:
    • The UK National Cyber Security Centre and its partners say changes to data, models or prompts can change a system's behaviour, and that major updates should be treated like new versions.
    • NIST says the data an AI system works with can change over time, sometimes significantly and unexpectedly, and that datasets can become stale.
    • The 2015 paper names "Changing Anything Changes Everything": a change in one input or setting can change the behaviour of the whole system.
    • The same paper says a threshold set by hand may no longer be valid once the model updates on new data.
  5. Each result with its denominator ("out of how many"), and "cannot count yet" where the logs do not record it, as 300.5 taught. Keep each stage's result separate.
  6. Not yet. Finding the right page and giving the right answer are different stages. The assistant can retrieve the new page, the allowed/denied pair can still pass, and the answer can still say 12 months. Only a test that grades the answer against the new fact, 24 months, catches that. A retrieval pass says nothing about the answer.

Exercise 3 (300.9-c)

Fix a receipt, then write the incident note. Here is a draft receipt from the practice log:

what: clear-claim C-88213
reviewer: Jordan Lee, jordan.lee@home-mail.example, on sick leave from Mon
approval: AP-2291
result: done
note: "customer says gearbox failed after 3 days, wants refund to card ending 4421"
  1. Name each line that leaks personal data or copies content, and say what it leaks.
  2. Rewrite the receipt with the course's five fields (what, who held it, which approval, result, how to undo), using references only. Add what else every receipt carries.
  3. An outside reader sees only your rewritten receipt. What can they tell from it, and what can they not? Who can check the rest, and how?
  4. In the same hour, 20 R1 claims were cleared after the budget broke, until H-017 paused the system by hand at 11:52. Nobody yet knows whether those 20 clearances were wrong, or why the stop did not act. Write a short incident note that answers the course's seven questions.
Model answer
  1. The reviewer line leaks an employee's name, home email and a health-related absence. The note line copies the customer's words and part of a card number. The UK National Cyber Security Centre says to treat logs as sensitive data and protect them, so a receipt points to records instead of copying them.
  2. For example:

     time: 2026-11-18 11:41:09
     agent: (the assistant's identity reference from 300.4, never its secret key)
     what: clear-claim C-88213
     who held it: holder H-017 (reviewer R1, directory ref P-0412)
     which approval: AP-2291, expires 2027-02-01
     result: done
     how to undo: reopen C-88213 (holder only)
     note: see claim record C-88213

    (The time and expiry date are invented practice data.)

  3. They can read what the gate recorded: this action, under this holder and this approval reference, with this result, without seeing anyone's personal details. They cannot tell from the receipt alone that AP-2291 really exists, covers clearing claims, or was still valid at that minute; a made-up receipt could quote a reference just as neatly. An authorised checker, such as an auditor, follows the references into the approval record and the trace, or relies on an attestation from someone they trust. The receipt is an index, not proof. Signing it would let a stranger check it has not been changed since it was signed, and which key signed it; that still does not show the approval was real, in scope or in date. Its references still point to real people, so it stays under access control and the company's retention and privacy rules.
  4. For example: "What happened: at 11:41 the per-reviewer budget broke for R1 (51 of 50). How caught: the monitor's budget alert paged holder H-017 at 11:41. What was stopped: the stop did not fail closed; 20 further R1 actions ran until 11:52, when H-017 paused the system by hand. Harm: not yet known; the 20 claims are being re-reviewed. Why: not yet known; being investigated. What changes: the stop is wired to refuse on the alert, and the stop runs are re-run and must pass before H-017 restarts it (holder H-017). Receipts: C-88219 to C-88502 and the dry-run traces." It names references, not claim details or people's personal data, and it says "not yet known" rather than guessing the harm or the cause.

Next module: 300.R Building in your job (role tracks and prep rails)