Level 300: Building with gen AI · Module 300.8

At the café counter, a man in an apron hands a brass key with a blank tag to a colleague, beside a blank clipboard. The chalkboard reads “Who holds it?”
Who holds it?

Hold: the operate gate · Standard gear

Reviewed and published, version 1.0.0. As of 19 Nov 2026. It is not a credential. Big Widget Company, its people and its systems are made up. The court cases are real and are given as allegations, not findings. This module does not say how either case ended: that is not checked.

You will be able to:

This module teaches the operate gate, the course's version of the Practice 300 Hold. You will use the same gate in module 300.9 (building the monitor and stop) and in project 300.P (a dry run on the course's synthetic files). The gate is the course's rule, not a law. Where a law says something similar, this module says which law and quotes it.

Where this sits in the three yeses

Module 100.9 taught the course's three yeses: Get to Yes, Stay at Yes, Recover to Yes. The operate gate is Stay at Yes. Writing the hold before launch gets the system to yes. That yes is conditional. It lasts only while the holder, the approval, the budget and the monitor are all still in place, and the gate checks them again before every action.

When the gate stops something, that is the start of Recover to Yes, not the end of it. A named person looks at what stopped and why, fixes it, and says yes again on the record. A paused system is not yet a recovered one.

The gate gives one of four answers, as in module 100.9: go ahead (allow), only this far (constrain), wait for a person, or no (deny). This module calls "wait for a person" a pause, so it does not get mixed up with the hold, which here means the written record below. The gate decides by its rules. The AI never releases its own pause. A named person does, on the record.

A pause also does not end by itself when the thing that caused it comes back. If the monitor goes down and comes back up, the system stays paused until the holder looks and records a release. The monitor coming back is a fact. Saying yes again is a person's decision.

1. Every action has a holder

Card 1, HOLDER · COURSE RULE: Name one person. Write who answers for this action and when their approval expires. Card 2, STOP · COURSE RULE: Fail closed. Holder, approval or record missing or expired? Do not run; page a person. Card 3, LEGAL CONTEXT [s6]: High-risk systems must be stoppable. Article 14: overseers must be able to intervene or halt it safely, as appropriate and proportionate.
The holder, expiry and fail-closed rule are course-owned procedure. Article 14 supplies the separate high-risk oversight context (s6); this summary does not show the whole operate gate or establish legal compliance.
Figure data as text

Who holds this action?

FocusLesson point
HOLDER · COURSE RULE · Name one personWrite who answers for this action and when their approval expires.
STOP · COURSE RULE · Fail closedHolder, approval or record missing or expired? Do not run; page a person.
LEGAL CONTEXT [s6] · High-risk systems must be stoppableArticle 14: overseers must be able to intervene or halt it safely, as appropriate and proportionate.

The holder, expiry and fail-closed rule are course-owned procedure. Article 14 supplies the separate high-risk oversight context (s6); this summary does not show the whole operate gate or establish legal compliance.

Objective: 300.8-a. Text equivalent of figures/who-holds-this-action.svg.

What "held" means

Modules 300.2 and 300.4 built a retrieval assistant and an agent with limits. Before either runs on its own at Big Widget, the course asks one question for each action it can take: who holds this?

A system is held when, for every action it can take, these four things are written down:

  1. Holder. One named person who answers for that action. A name, not "the team", "IT" or "the business". The holder is the one who said this action may run, who can stop it, and who is asked afterwards why it said yes. Alerts can still go to an on-call rota with an escalation route, so someone always picks up. The rota handles the alert. The holder still answers for the decision.
  2. Expiry. The date the holder's standing approval runs out. Approvals go stale: people change jobs, systems change, risks change. An approval with no end date is an approval nobody is checking.
  3. Stop. What makes the action stop, and the rule that it fails closed: if the holder, the approval, or the record of the approval is missing or out of date, the action does not run. It pauses and pages a person.
  4. Out-of-band check for anything irreversible and high-value: a second confirmation on a channel the person making the request does not control.

The standing approval says the system may do this kind of action at all. It is not a decision about any one case. Where module 300.4's authority table says an action needs an approver, the gate also needs that approver's named decision on this case. A current standing approval with no decision on the case means the action does not run.

Anything not on the list is not allowed (module 300.4's authority table). The course's practice prompts (levels/300/practice/hold-prompts.md) add four more parts: a budget, a judgment rule, a receipt and a coverage count. Sections 2 to 4 cover the budget and the judgment rule. The receipt and coverage are below, and module 300.9 builds them.

The misconception this corrects: "The team owns it, so no single name is needed." Shared ownership is fine for the work, and a team rota is fine for answering alerts. It is not fine for the answer to "who said this could run, and who stops it?"

Failing closed

"Fail closed" means that when something the gate depends on is missing, the safe answer is no. "Fail open" means the system keeps going because nobody told it to stop. For a held system:

What is missing or wrongFail open (wrong)Fail closed (right)
The holder has left the companyKeeps running on the old approvalPauses; pages the holder's manager to name a new holder
The approval expired yesterdayKeeps running until someone noticesPauses at expiry; pages the holder
An action has no approval recordRuns, and logs a warningDoes not run; the receipt says which record is missing
An approver action has no named decision on this caseRuns on the standing approvalDoes not run; pages the holder
The budget is exceeded, or the budget count cannot be readKeeps actingPauses; pages the holder; stays paused until the holder releases it
The monitor that checks the budget is downKeeps acting, unwatchedPauses; stays paused after the monitor is back, until the holder releases it on the record
The monitor comes back, but nobody has released the pausePicks up where it left offStill paused
The store that remembers the pause cannot be readCarries on once it can be read againPauses; notes the fault in a second, separate record; stays paused until the holder releases it
Neither record can be read or writtenCarries on once they recoverThe system that carries out the actions halts; only a named person restarts it, on the record

Take the quiz-style case: the approval for an agent's actions was given eight months ago by a manager who has since left. Is the system still held? No. The holder no longer exists, so no one answers for it, whatever the expiry date says. A fail-closed gate stops the actions that needed that approval and pages someone with the authority to name a new holder. It does not quietly carry on.

The receipt and the coverage count

Each time the gate answers, it leaves a receipt. In this course a receipt holds references, not content: which action, which hold entry (an id such as HOLD-WC-03), which approval record, which decision record, the gate's answer, and how to undo it. It never holds the claim text, the letter, a customer's details, or a person's name or login, the holder's included. Those stay in the controlled hold and work records the receipt points to.

A reference keeps the name out of the receipt. It does not make the receipt anonymous: anyone allowed to open the hold can see that HOLD-WC-03 is Dana's. Whether that counts as personal data under a particular law is a question for the organisation's privacy owner; this module has not checked it.

The coverage count asks, of the actions the system actually took, how many the gate checked and allowed first, matched one action at a time. Two totals side by side are not enough: the gate could have looked at ten requests and turned them all down while the system ran ten others with no check at all. Actions that ran with no check, or ran after the gate said wait or no, are listed one by one as breaches. They are not a smaller percentage. Requests the gate turned down are counted separately, because they never ran. Every run counts, not just the first: if one action runs twice, each run is checked against what the gate said just before it, so a second run after a pause shows up as a breach. An action with more than one run is also listed, so a person can find out whether it really ran twice or was only logged twice. Matching needs two things in every record: the action's id, and its place in one shared log that puts every check and every action in order. A clock time is not enough, because two records can share one. If an id is missing, or two records claim the same place, the hold says "cannot count yet". It does not guess, and it never treats records with no id as one action.

Why an irreversible, high-value action needs a second channel

In June 2024 the Hong Kong Government told its Legislative Council about a case the Police received in end-January 2024. Fraudsters used deepfake technology to "fabricate a pre-recorded video conference". The person targeted, whom the record calls "the informant", "eventually gave authorisation to transfer funds to five local bank accounts" and "suffered a loss of about HK$200 million".1 The record adds: "Since the meeting was pre-recorded, there was actually no interaction between the informant and the fraudster."1 (The government record does not name the organisation, and this course does not either.)

The lesson for a hold: a face or a voice on a call is not identity. In general advice in the same reply, not tied to this case, the Police told the public to verify, including "verifying with a phone call when a relative or a friend makes a request for remittance".1 The course's gate turns that into a broader rule of its own: an irreversible, high-value action needs confirmation on a channel the requester does not control, such as calling back on a number already on file. The confirmation must be about this payment: this payee, this amount. A tick in a box that anyone could set is not a confirmation. Money that has left the company cannot be recalled by changing a setting.

Human oversight in law

The EU AI Act sets a human-oversight duty for high-risk AI systems. Article 14 says they must be designed so "they can be effectively overseen by natural persons during the period in which they are in use".2 The people assigned to oversee them must be enabled, "as appropriate and proportionate", among other things, "to intervene in the operation of the high-risk AI system or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state."2 Whether and when the Act applies to a given Big Widget system is module 300.7's question. The point here is that the law's idea of oversight includes a person who can actually stop the system, safely. This module does not claim that the gate on its own meets the Act.

At Big Widget Company

Big Widget's warranty-claims assistant reads incoming claims and flags the ones it thinks the warranty does not cover. Reviewers then clear or reject the flagged claims. Its hold, in part:

ActionHolderExpiresStop (fails closed)
Flag a claim for reviewDana Okafor, head of warranty operations31 May 2027No current approval or holder: flagging pauses; claims go to the normal queue unflagged
Send a rejection letterDana Okafor31 May 2027No reviewer's named decision on this claim: letter is not sent, even with Dana's standing approval current
Refund over $10,000 to a dealerSam Reyes, finance controller28 Feb 2027No call-back confirmation of this refund, on the dealer's number on file: refund is not paid
Change a dealer's bank detailsNever by the assistant—Not on the list, so denied

Alerts for the warranty actions go to the warranty on-call rota, which escalates to Dana if nobody answers. Dana remains the holder. Only Dana, or someone she names on the record, releases a pause on her actions; Sam does the same for refunds.

2. Small acts add up: aggregation

The warning: the alleged Cigna totals

Each single action can look low-risk. In volume, the same action can become a high-stakes act. The course calls this aggregation.

The warning case is Cigna's PXDX process. In March 2023, ProPublica reported, citing "corporate documents and interviews with former Cigna officials", that "Over a period of two months last year, Cigna doctors denied over 300,000 requests for payments using this method", "spending an average of 1.2 seconds on each case".3 Asked whether PXDX lets its doctors reject claims without examining them, Cigna said that description was "incorrect". (The word "incorrect" answered that question about how claims are reviewed. It was not said of the totals.) It said the process exists to "accelerate payment of claims for certain routine screenings", and that "Reviews occur after the service has been provided to the patient and does not result in any denials of care".3

In July 2023 a class action, Kisting-Leung v. Cigna Corp., was filed in federal court in California. The complaint alleges that "over a period of two months in 2022, Cigna doctors denied over 300,000 requests for payments using this method, spending an average of just 1.2 seconds “reviewing” each request", and its footnote for that sentence cites the ProPublica report.4 So the court filing is not a second, independent count: it repeats the reporting. Both are claims, not findings. This module does not say how the case ended.

Use these figures only as a warning, as alleged totals. They are not inputs to the arithmetic below, and this course does not turn them into an hourly rate. The practice uses an invented log.

Setting a budget on a synthetic log

A budget is the most actions allowed in a time bucket, and what happens when it is exceeded. A good budget has two lines:

It is checked against each bucket (here, each hour), not against an average, because an average spreads one bad hour across the good ones.

Here is the course's synthetic log for Big Widget's warranty-claims assistant. Reviewers R1 to R3 clear the claims it flags; each number is how many decisions a reviewer made in that hour. The numbers are invented and say nothing about any real company. The log shows what happened with no live stop running: it is what you would find if you looked back.

HourR1R2R3System total
09:0012101133
10:001491336
11:0071121093
12:00811928

The budget: at most 50 reviewer decisions per reviewer per hour, and 120 system-wide per hour.

What the stop does

The budget counts decisions, so the stop must stop decisions, not just new work arriving. Pausing new flags to R1 would not be enough: R1 could keep clearing cases already waiting in the queue and go past 50 anyway.

With a live gate, R1's 51st decision in the hour is the one that pauses:

  1. R1's further decisions in that hour do not take effect. No rejection letter goes out on them. They wait for a person.
  2. Cases already in R1's queue stay where they are. Nothing is approved or rejected automatically.
  3. The gate pages Dana. She can move the waiting cases to a reviewer who is under budget, or leave them to wait. She records which she chose.
  4. New flags stop going to R1 until Dana releases the pause.

A new hour starting does not end the pause. The count starts again at noon; the pause stays until Dana releases it. If the system line is crossed, the same happens for everyone, and Dana decides what moves first. With the stop running, the log would show 50 for R1 at 11:00, not 71. The synthetic log is for practising detection; the stop is what turns detection into enforcement.

The misconception this corrects: "Each decision is low-risk, so the system is low-risk." Volume changes the risk. A budget makes the volume visible, and a stop on the counted action keeps it in bounds.

3. A signature is not a review: the rubber stamp

What a rubber stamp is

A rubber-stamp review is one where a person signs off without actually reviewing: the name is on the decision, but the judgment came from the system. It looks like human oversight. It is not.

The Cigna complaint alleges exactly this shape. It says "Cigna's doctors instantly reject claims on medical grounds without ever opening patient files", and that Cigna "allowed its doctors to sign off on the denials in batches".4 ProPublica quotes a former Cigna doctor: "We literally click and submit" and "It takes all of 10 seconds to do 50 at a time."3 Cigna disputes the description.3 Whether or not those allegations are proved, they describe what a rubber stamp looks like in the records.

Why it is not oversight

The EU AI Act's oversight article names the problem. People assigned to oversee a high-risk system must be enabled "to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)", in particular where the system gives "information or recommendations for decisions to be taken by natural persons".2 They must also be able "to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output".2

A rubber stamp fails that test in practice even when it passes on paper. A person who never sees the case cannot know when to override. A person given too little time, or punished for disagreeing, will not override. The Lokken complaint (section 4) alleges both: that the insurer set "targets to keep stays at skilled nursing facilities within 1% of the days projected by the AI Model", and that "Employees who deviate from the nH Predict AI Model projections are disciplined and terminated".5 Those are allegations. They show what to check for.

The misconception this corrects: "A human in the loop means a human decided." A human decided only if they had the information, the time and the authority to decide otherwise, and used them.

The evidence that tells you

You cannot see judgment, but you can see its traces. For a batch of reviewed decisions, ask the logs:

QuestionReal review looks likeRubber stamp looks like
Did the reviewer see the case evidence?A recorded view of the case for each decision, by whatever route it reaches themNo views recorded, or one per batch
How long per case?Varies with the caseSeconds each, nearly identical
How often does the reviewer disagree with the system?Sometimes, with a reason recordedNever, out of hundreds
Can the reviewer say no without penalty?Yes, and it is written downTargets or pressure to match the system
Signed one at a time or in a batch?One at a timeWhole batches with one action

Read these together, not one at a time. None of them proves a rubber stamp on its own, and none proves real review. Opening a file shows it was opened, not that it was understood. A low disagreement rate can mean the system is usually right. There is no target rate of disagreement: a reviewer who disagrees to hit a quota is not reviewing either. The traces tell you where to ask questions. A person reads them and decides.

Report each of these with its denominator ("0 overrides out of 412 decisions"), and say "cannot count yet" when the log does not record it. If the log has no record of case views at all, you cannot tell a real review from a rubber stamp, and the hold should say so. A missing record is not a zero.

4. Allegation, statement, gate: the nH Predict case

Three different things

The nH Predict case is the course's practice in keeping three things apart.

What the complaint alleges. On 14 November 2023, the estates of Gene B. Lokken and Dale Henry Tetzloff filed a class action, Estate of Gene B. Lokken v. UnitedHealth Group, Inc., in federal court in Minnesota, against UnitedHealth Group, UnitedHealthcare and naviHealth.5 It alleges the "illegal deployment of artificial intelligence (AI) in place of real medical professionals to wrongfully deny elderly patients care owed to them under Medicare Advantage Plans by overriding their treating physicians' determinations as to medically necessary care based on an AI model that Defendants know has a 90% error rate."5 It names the model "nH Predict", used for post-acute care, and alleges coverage decisions were made "not based on individual patient's needs, but based on the outputs of the nH Predict AI Model".5 The 90% figure is the complaint's allegation. Its basis is not checked.

What the company said. The same day, STAT reported an emailed statement from a UnitedHealth spokesperson, who said that the naviHealth predict tool is not used to make coverage determinations.6 The statement said: "The tool is used as a guide to help us inform providers, families and other caregivers"; "Coverage decisions are based on CMS coverage criteria and the terms of the member's plan"; and the "lawsuit has no merit and we will defend ourselves vigorously".6

The course's gate. A model may inform but not replace clinical judgment in a coverage decision. This is the Foundation's teaching rule, from the course's practice prompts. It is not a finding about the case, it is not something the company said, and it is not a quote from any law.

A lawsuit is not a finding. A complaint is one side's account, filed to start a case. A company statement is the other side's account. This module states neither as fact, and does not say how the case ended.

Why the gate asks the same thing whoever is right

You do not need to know who is right to know what the hold requires.

Either way, the gate asks the same thing: when the model's predicted stay and the treating clinician disagree, a qualified person reviews that patient's case and decides, and the record shows who and why. The gate does not say whose figure must win. It says a qualified person, not the model, makes the call.

The misconception this corrects: "If the model is accurate on average it can decide each case, and a lawsuit is a finding." An average can inform a case, but it does not decide the patient in front of you; this is the aggregation problem in section 2, turned around. And a complaint is an allegation until a court decides.

At Big Widget Company

Big Widget's service planner predicts how many repair hours a warranty job should need. The service engineer who has examined the widget sometimes needs more. The hold for "approve repair hours":

Key words

Your check

Before you call a system held, you should be able to say each of these in the first person, or say honestly that you can't yet:

"I can't say yet" is an honest answer. It comes with a next step: who will find out, and by when.

Further learning (free university courses)

These are optional background, not sources for this module.

Sources

  1. s5 Government of the Hong Kong Special Administrative Region (written reply by the Secretary for Security to a Legislative Council question) (2024). LCQ9: Combating frauds involving deepfake. Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, fresh session separate from the writer, reviser and earlier checkers), round 3 re-check, 19 Nov 2026; live page, info.gov.hk, full reply. ↩ ↩ ↩
  2. s6 European Parliament and Council of the European Union (text as shown by the European Commission's AI Act Service Desk) (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 14 Human oversight. Standard. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, fresh session separate from the writer, reviser and earlier checkers), round 3 re-check, 19 Nov 2026; AI Act Service Desk Article 14 page (consolidated text as at 27 Jul 2026) and the Official Journal text (OJ L, 12.7.2024) via Publications Office Cellar DOC_1; every quoted passage matches in both. ↩ ↩ ↩ ↩
  3. s2 Patrick Rucker (The Capitol Forum), Maya Miller and David Armstrong (ProPublica) (2023). How Cigna Saves Millions by Having Its Doctors Reject Claims Without Reading Them. Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, fresh session separate from the writer, reviser and earlier checkers), round 3 re-check, 19 Nov 2026; live page, whole page saved. ↩ ↩ ↩ ↩
  4. s1 Plaintiffs Suzanne Kisting-Leung and Ayesha Smiley, by Clarkson Law Firm, P.C. (pleading filed in the United States District Court for the Eastern District of California, via CourtListener) (2023). Kisting-Leung v. Cigna Corp., No. 2:23-cv-01477 (E.D. Cal.), class action complaint (docket entry 1). Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, fresh session separate from the writer, reviser and earlier checkers), round 3 re-check, 19 Nov 2026; RECAP PDF Doc. 1, full text. ↩ ↩
  5. s3 Plaintiffs the Estate of Gene B. Lokken and the Estate of Dale Henry Tetzloff (pleading filed in the United States District Court for the District of Minnesota, via CourtListener) (2023). Estate of Gene B. Lokken v. UnitedHealth Group, Inc., No. 0:23-cv-03514 (D. Minn.), class action complaint (docket entry 1). Record. Checked 2026-11-19 by source-checker agent (Claude Opus 5.5, fresh session separate from the writer, reviser and earlier checkers), round 3 re-check, 19 Nov 2026; RECAP PDF Doc. 1, full text. ↩ ↩ ↩ ↩
  6. s4 Casey Ross and Bob Herman (STAT) (2023). UnitedHealth faces class action lawsuit over algorithmic care denials in Medicare Advantage plans. Record. Checked 2026-11-20 by source-checker agent (Claude Opus 5.5, fresh session separate from the writer, reviser and earlier checkers), round 4 re-check, 20 Nov 2026; live page (HTTP 200), full article body, all nine uses including the revised plain.md wording. ↩ ↩ ↩

Next: the quiz