
Hold: the operate gate · Plain gear
Reviewed and published, version 1.0.0. As of 19 Nov 2026. It is not a credential. Big Widget Company and everyone in it are made up. The court cases are real. What they say are claims, not proven facts. We have not checked how the cases ended.
What you will learn:
- how to give every action a system takes one named person, an end date and a stop (300.8-a);
- why lots of small actions can add up to a big one, and how to set a limit (300.8-b);
- how to tell a real review from a "rubber stamp" (300.8-c);
- how to keep a lawsuit's claims, the company's answer and our own rule apart (300.8-d).
Where this fits
In module 100.9 you met the three yeses: Get to Yes, Stay at Yes and Recover to Yes. This module is about Stay at Yes. Saying "yes" once, before a system starts, is not enough. The "yes" has to keep being true while it runs.
A gate gives one of four answers: go ahead, only this far, wait for a person, or no. In this module we call "wait for a person" a pause. The AI never ends its own pause. A named person does.
A pause doesn't end by itself, either. Say the alarm system breaks and the gate pauses. Then the alarm system is fixed. The pause stays on. The holder looks at what happened and says "go" again, on the record. Fixing the problem is one thing. Saying "yes" again is a person's job.
1. Who is holding it?
Figure data as text
Who holds this action?
| Focus | Lesson point |
|---|---|
| HOLDER · COURSE RULE · Name one person | Write who answers for this action and when their approval expires. |
| STOP · COURSE RULE · Fail closed | Holder, approval or record missing or expired? Do not run; page a person. |
| LEGAL CONTEXT [s6] · High-risk systems must be stoppable | Article 14: overseers must be able to intervene or halt it safely, as appropriate and proportionate. |
The holder, expiry and fail-closed rule are course-owned procedure. Article 14 supplies the separate high-risk oversight context (s6); this summary does not show the whole operate gate or establish legal compliance.
Objective: 300.8-a. Text equivalent of figures/who-holds-this-action.svg.
Think of a dog on a lead. Someone has to hold the lead. Not "the family". One person, right now.
AI systems at work are a bit like that. Big Widget Company has a warranty assistant. It reads repair claims from dealers. It marks the ones it thinks the warranty does not cover. Then people check them.
Before it runs on its own, we ask about each thing it can do: who is holding this?
For each action, we write down four things:
- Who holds it. One named person. Not "the team". The team can share the work. A team rota can answer the alarm at 3 a.m. But one named person said "yes" to this action. That person can stop it, and has to explain the "yes" later.
- When it runs out. The date the "yes" ends. People change jobs. Systems change. A "yes" with no end date is a "yes" nobody is watching.
- What stops it. And this rule: if anything is missing, it stops. If the person has left, it stops. If the "yes" ran out, it stops. If there is no record of the "yes", it stops. This is called failing closed. Like a fire door that shuts when the power goes off.
- A second check for big, can't-undo actions. Check on a different line, one the person asking does not control.
One more thing. The holder's "yes" means the assistant may do this kind of thing. It is not a "yes" to each case. Some actions, like sending a letter that turns down a claim, also need a reviewer's named decision on that claim. No decision on record? The letter does not go.
Here is a puzzle. A manager said "yes" to the assistant eight months ago. Then the manager left the company. Is the assistant still being held? No. Nobody is holding the lead. So it should stop and call someone who can name a new holder. It should not just keep going.
The receipt
Every time the gate answers, it leaves a receipt. The receipt says what was asked, which hold it used, which records it checked, and what the answer was. It points to the hold by a code, like HOLD-WC-03. It does not hold anyone's name, private details, the claim, or the letter. Those stay in the work records, which are kept locked up. The receipt only points to them.
A code is not a disguise. Someone allowed to open the hold can still see whose it is. That's fine. The point is that the receipt itself carries no names.
The hold also counts the gate's work. Of the actions that really happened, how many did the gate check and say "go" to first? Match them up one by one. Don't just put two totals side by side. If something ran with no check, or ran after the gate said "wait" or "no", list it by itself. That's a problem to fix, not a smaller score. Count every run, not just the first. Say the gate says "go", the action runs, then the gate says "wait", and the action runs again. That second run is a problem too, so it gets listed. If we can't match them up yet, we say "we can't count that yet". We don't guess.
The fake video call
In June 2024, the Hong Kong Government described a real case to its lawmakers. It came to the Police at the end of January 2024. Crooks used deepfakes (fake video made with AI) to make a fake video meeting. It was recorded ahead of time. The person tricked by it moved money to five bank accounts. The loss was about 200 million Hong Kong dollars.1 The meeting was not even live, so nobody on it was really talking to the victim.1 (The government did not name the company. We don't either.)
A face on a screen is not proof of who someone is. The Hong Kong Police, in general advice, tell people to check with a phone call when a relative or friend asks them for money by video or voice message.1 Our own rule goes further. Before any big payment that can't be undone, call back on a number you already had. Don't use the one in the message. And check that the call is about this payment: who gets it, and how much.
What the law says
A European law, the EU AI Act, has a rule for "high-risk" AI systems. People must be able to watch over them while they run.2 And, where it fits the system, those people must be able to stop it, with a "stop" button or something like it, so it comes to a halt safely.2 Module 300.7 covers when that law applies. The idea is like ours: someone holds it, and someone can stop it.
Myth: "The team owns it, so we don't need one name." Truth: The team can do the work, and a rota can answer the alarm. One named person answers for the "yes" and can stop it.
2. Small things add up
One raindrop is nothing. A million raindrops is a flood.
A single "no" to one claim can look small. But a system can say "no" thousands of times a day. Then it is not small any more.
A warning from a real case
In 2023, a news group called ProPublica reported on the health insurer Cigna. It said Cigna's doctors had turned down over 300,000 payment requests in two months. It said they spent about 1.2 seconds on each one, on average. It said this came from company papers and former staff.3 Cigna said it was not correct that its doctors turn claims down without looking at them. It said the checks happen after the care is given, and do not deny anyone care.3
Later that year, people sued Cigna in a US court. Their complaint repeats the same numbers, and points to ProPublica for them.4 So it is not a second count. A complaint is one side's story. A court has to decide if it is true. We use these numbers only as a warning, not for our maths.
Set a limit
A budget is a limit: the most actions allowed in one hour. It needs two parts:
- a limit for each person, because one person can be swamped while the total looks fine;
- a limit for the whole system, because many people can add up to too much.
And you check every hour, not the average. An average hides the worst hour.
Here is a made-up log from Big Widget. Three reviewers, R1, R2 and R3, decide on claims the assistant marked. Each number is how many claims a reviewer decided that hour. The numbers are invented for practice. Nothing was stopping anyone when this log was made.
| Hour | R1 | R2 | R3 | All |
|---|---|---|---|---|
| 9 a.m. | 12 | 10 | 11 | 33 |
| 10 a.m. | 14 | 9 | 13 | 36 |
| 11 a.m. | 71 | 12 | 10 | 93 |
| 12 noon | 8 | 11 | 9 | 28 |
The limit is 50 decisions per person each hour and 120 for everyone each hour.
- At 11 a.m., R1 decided 71. That is over 50.
- The busiest hour for everyone was 93. That is under 120. So if you only had the "everyone" limit, you would miss R1.
- Spread over four hours and three people, it is about 16 each hour. That looks fine. The average hides the bad hour.
What the stop does
We are counting decisions. So the stop has to stop decisions. Just sending R1 no new claims is not enough. R1 could still work through the pile already waiting and go over 50.
So, if the stop had been running, at R1's 51st decision that hour:
- R1's next decisions wait. No letters go out on them.
- The claims in R1's pile stay put. None is approved or turned down by the machine.
- Dana, the holder, gets a call. She can hand the waiting claims to someone under the limit, or let them wait.
- R1 gets no new claims until Dana says so.
- When noon comes, the count starts again. The pause does not. It waits for Dana.
Myth: "Each decision is small, so the system is safe." Truth: Lots of small decisions can add up to a big one. A limit lets you see it. A stop on the same decisions keeps it in bounds.
3. A signature is not a check
Imagine a teacher who signs every homework sheet "checked" without reading any of them. Her name is on every sheet. But nobody checked anything.
That is a rubber stamp. A person signs off, but they did not really look.
The people who sued Cigna say this happened there. Their complaint says doctors turned claims down without ever opening the patient's file, and signed off on them in batches.4 A former Cigna doctor told ProPublica: "We literally click and submit."3 Cigna says the reporting was wrong.3 We don't know who is right. But the claim shows what a rubber stamp looks like.
The EU AI Act warns about this too. It says people overseeing a high-risk system must stay aware of the pull to just trust the machine. It calls this automation bias.2 They must also be free to ignore or overrule what the system says.2
How can you tell a real check from a rubber stamp? Look at the records:
- Did they see the case? A real check looks at it. But opening a file is not the same as reading it.
- How long did it take? A few seconds each, every time, is a warning sign.
- Do they ever disagree with the system? "Never, out of hundreds" is a warning sign. But there is no right number. Disagreeing just to hit a number is not checking either.
- Are they allowed to say no? If people get in trouble for disagreeing, they won't.
- One at a time, or a whole pile at once? A whole pile with one click is a warning sign.
Warning signs are questions to ask, not proof. A person looks at them all together and decides. If the records don't show these things, say so: "We can't tell yet." No record is not the same as zero.
Myth: "There's a human in the loop, so a human decided." Truth: A human decided only if they had the facts, the time and the power to say no.
4. Claim, answer and our rule
Here is another real case. In November 2023, a group lawsuit was filed in a US court against the health insurer UnitedHealth and two related companies.5 The people suing say an AI model called nH Predict was used to cut off recovery care for older patients, such as stays in nursing homes. They say it overruled what the patients' own doctors decided. They say the model is wrong 90% of the time.5 These are their claims. We have not checked the 90% number.
The same day, a UnitedHealth spokesperson told the news site STAT that the tool is not used to decide coverage.6 The company said the tool is "a guide" to help inform care providers, families and other carers. It said its decisions follow the Medicare agency's coverage rules and the patient's plan. It said the lawsuit "has no merit".6
So there are three different things here. Keep them apart:
| What it is | Example |
|---|---|
| A claim in a lawsuit | "The model overruled the doctors." |
| The company's answer | "The tool is a guide. Coverage follows the rules and the plan." |
| Our rule (the course's, not a law, a court or the company) | A model may help, but a qualified person looks at the case and decides. |
A lawsuit is not a verdict. It is one side's story.
Why our rule works either way
- If the people suing are right, the model replaced the doctors' judgment. Our rule says that is not allowed.
- If the company is right, the tool is only a guide. That fits our rule. STAT also reported that the company has said its own doctors make the final call on each payment denial. And STAT said those doctors work from suggestions to deny payment, made by staff who have a 1% target to meet.6 Those are both accounts, not records. The records should be able to show who looked at each case.
Either way, the rule is the same. When the model and the doctor disagree about a patient, a qualified person looks at that patient's case and decides. The record says who decided and why. Our rule does not say whose number wins. It says a person, not the model, makes the call.
At Big Widget, a planning tool guesses how many hours a repair should take. The engineer who has looked at the widget sometimes needs more. Big Widget's rule for its own repairs: the tool helps, the engineer decides, and the reason goes in the record. If engineers never go against the tool, out of 2,000 jobs, that is a question to ask, not a reason to cheer.
Myth: "If the model is right on average, it can decide every case. And a lawsuit proves what happened." Truth: Right on average can still be wrong for this person. And a lawsuit is a claim until a court decides.
Words to know
- Hold: one named person answers for an action, until a date, and can stop it.
- Fail closed: if something is missing, it stops.
- Pause: the gate says "wait for a person". A person ends it, never the AI. It doesn't end by itself when the problem goes away.
- Budget: the most actions allowed per person and in total each hour.
- Receipt: a note of what the gate decided. No names, no private details.
Your check
Can you say these? If not, "not yet" is an honest answer. Then say who will find out.
- I can name who holds each action, and when their "yes" runs out.
- I know what stops each action, and that only a person can restart it.
- I check the busiest hour, not the average.
- I know how many actions the gate checked, out of how many ran, or I know that I don't.
- Rubber stamp: signing off without really checking.
- Allegation: what one side claims in a lawsuit. Not proven.