
Evaluating what you built · Plain gear
You will be able to: test each part of your AI build on its own, and count each part's results out of the right number; check an AI marker against a person before you let it mark your work; and say the honest number, even when the honest answer is "we can't count that yet".
This module is as of November 2026. Reviewed and published, version 1.0.0. It does not earn you a credential. Big Widget Company is made up, and so are all its numbers here.
1. Test each part on its own
The shopper and the cook
Think of a kitchen with two jobs. A shopper fetches the ingredients. Then a cook makes the dish.
If dinner tastes wrong, whose fault is it? Maybe the shopper brought the wrong things. Maybe the shopper got it right and the cook made a mess. If you only taste the dinner, you can't tell.
An AI assistant that looks things up works the same way. First it finds passages in a set of documents. Then the AI writes an answer from them. An AI agent adds one more job: it asks to use tools, like opening a ticket, and the system decides yes, wait for a person, or no.
The people who made a well-known way to test these assistants, called RAGAs, say the testing is hard because there are several things to check. Does the search find the right passages? Does the AI use them faithfully? Is the writing good?1 So they check each one on its own.1
Six things to count
In Level 200 you made a small test: real-looking questions, each with a known good answer. Now you keep that test, but you look at each step, not just the final answer.
- Finding. Did the search bring back the passage with the answer?
- Sticking to it. Did the answer only say things the passages back up?
- After a miss. When the search missed, did the AI say "I didn't find it", or answer anyway?
- Saying "I don't know". When the answer isn't in the documents at all, does it say so?
- Waiting. When the rules say a person must approve, does the agent wait?
- Stopping. When someone hits stop, does it really stop?
Each one gets its own "out of how many".
Here is the tricky one. You judge "sticking to it" only for questions where the search found the right passage. That way a search mistake isn't counted as a writing mistake.
But that doesn't let the cook off the hook. If the shopper brings the wrong things, a good cook says "I can't make this dish." A bad cook serves something else and calls it dinner. So check that too: out of the times the search missed, how often did the AI answer anyway instead of saying so?
Answering anyway is not always making things up. Say someone asks the price of part A, but the search only found the page about part B. The AI says "Part B costs \$5". That is true, and the page backs it up. But it doesn't answer the question, so it still fails. If you also want to know how often the AI made things up after a miss, check those answers against the pages it found. Count that on its own.
One question can go wrong in two places. You don't have to pick just one to blame. Each check has its own count.
You still want the final score: how many final answers were right and useful? But you get it by marking each final answer, the way you did in Level 200. You can't get it by taking away the other counts. An answer can stick to its passages and still not answer the question.
A worked example
Big Widget's dealer team built an assistant for warranty rules. It can also open a service ticket. They test it with 40 questions. Every answer is somewhere in their documents. They also run 12 "must wait" tests and 4 "stop" tests.
- Finding: the search missed the right passage 6 times out of 40.
- Sticking to it: that leaves 34 questions where the search worked. The answer said something the passages didn't back up 5 times out of 34.
- After a miss: each of the 6 times the search missed, the AI answered anyway: 6 out of 6. It never said "I didn't find it". The team checked those 6 answers too. Each one made up a warranty rule the pages didn't back up.
- Final answers: a person marked every final answer. 29 out of 40 were right and useful.
- Waiting: the agent skipped a needed approval 1 time out of 12.
- Stopping: the agent kept going after stop 1 time out of 4.
The 29 out of 40 tells you something is wrong. It doesn't tell you what. The step counts do. Six misses mean: fix the search or the documents. Made-up answers, after a miss and when the passage was there, mean: fix how the AI writes, so it says when it hasn't found the answer. And the skipped approval and the missed stop must be fixed before it goes live, however good the answers are.
If the search never misses in a test, the "after a miss" check has nothing to count. Zero out of zero is not a perfect score. It means you haven't tested that yet. Add questions the search is likely to miss.
The US standards agency NIST says the same. Write down your tests and how you scored them.2 Test in conditions like the real job.2 Check the sources the AI points to.3
2. Check the AI marker before it marks your work
Figure data as text
Calibrate before grading
- SET · course step · Write the rule first — set the false-accept limit and minimum sample before grading; include hard cases.
- GRADE [s3, s4] · An independent person grades — use the rubric without seeing the model grader’s score; then run the grader.
- COMPARE · course step · Count agreements and errors — a false accept passes an answer the person failed; count false rejects too.
Objective: 300.5-b. Text equivalent of figures/calibrate-before-grading.svg.
A helper who marks homework
Marking hundreds of answers by hand is slow. So teams often use an AI to mark the answers. It's like a teacher getting a helper to mark homework.
A big study in 2023 found that strong AI markers agreed with people more than 8 times in 10. That is about as often as people agree with each other.4 Good news. But that was on the study's own questions.4 It doesn't tell you how well your AI marker marks your answers.
The same study found AI markers have habits:4
- They can favour whichever answer comes first.
- They can favour longer answers, even when those aren't as clear.
- They can favour answers written by themselves.
- They can be weak at maths. Big Widget answers are full of numbers, so that matters.
One fix from the study: ask twice, swapping the order of the two answers. Only count a win if the same answer wins both times.4
The RAGAs makers tested their own AI marker against people too. It matched people more often on some kinds of check than others. They said one kind was "the hardest" to mark.1 Even the inventors' own marker was uneven.
Check the helper against the teacher
Before you trust the helper, check it.
- Write your rule first. For example: "out of every 8 answers a person says are wrong, the AI marker may let through at most 1." Write it before you look, so nobody can move the line later.
- A person marks a pile of answers. They don't peek at the AI's marks. It should be someone who didn't build the system. NIST says checks by people who weren't the builders help.23
- The AI marks the same pile.
- Count the misses that matter. The worst one is when the person says "wrong" and the AI says "fine". That's a wrong answer that would slip through. Call it a false accept.
A worked example
The warranty team takes 30 answers. The person marks 8 of them wrong. The AI marker agrees about most answers. But it says "fine" to 2 of those 8 wrong ones.
The rule allowed 1 out of 8. The AI let through 2. So the AI marker can't be the one that says the assistant is ready. A person marks it instead, or the team fixes the AI marker and checks it again.
Who decides that? The person in charge of the launch, using the rule they wrote first. The AI marker doesn't get to decide whether it can be trusted.
Watch out for one thing. The AI and the person agreed on 26 of the 30 answers. That sounds great. But it hides the 2 wrong answers that would have slipped through. Always count the false accepts on their own.
And if your pile has too few answers the person marked wrong, you can't judge the marker yet. Mark more.
The person has to really look
A check against a person only works if the person really reads. In 2023 two people insured by Cigna, a health insurer, sued it. Their court papers claim that Cigna's doctors turned down more than 300,000 payment requests in two months in 2022. They claim the doctors spent 1.2 seconds on each, on average, and signed them off in batches.5 These are claims in a lawsuit, not proven facts. The numbers come from news reporting that the papers point to.5 We have not checked how the case ended.
The lesson still holds. Clicking "fine" in a second is not checking. Write down who marked each answer, and when, in the team's own records. That shows someone was there. It doesn't prove they read it.
3. Say the honest number
Four questions before you share a number
In Level 100 you learned to ask "out of how many?" about other people's claims. Now ask it about your own. Before you share a number, say:
- What did you count? Questions? Answers? Tool uses?
- Out of how many? And did you leave any out?
- Who marked them? A person, or an AI marker that passed its check?
- When, and which version? AI tools and documents change.
NIST says to test before launch and keep testing after.2 It also warns not to judge a system from a few quick tries, and says lab tests can miss what happens in real life.3
"We can't count that yet"
Sometimes you can't count something. Maybe the live assistant's record keeps the question and the answer, but not which passages it found. Then you can't count how often the search missed in real use.
Don't guess. Say "we can't count that yet". Say why, and say what you'd need to record to count it. NIST says the same: write down what you can't measure, and why.23
A worked example: "95% accurate"
A slide says: "Warranty assistant: 95% accurate." Behind it is a made-up record of 200 questions. Dealers gave 190 of the answers a thumbs-up.
- Ask: 95% of what? Out of how many? Who marked them? Here it's 190 thumbs-up out of 200. That shows dealers were happy. It doesn't show the answers matched the warranty rules.
- You can count: 190 thumbs-up out of 200.
- You can't count yet: how many answers were really right, or how often the search missed. The record doesn't keep that. To count it, keep the passages it found, and have a person mark some answers.
The Cigna numbers are the same kind of puzzle. "More than 300,000 turned down" is a claimed count.5 On its own it doesn't say how many were wrong, or out of how many requests. So the honest answer is "can't count that from these numbers".
The honest report
Tested on 18 Nov 2026. Search missed 6 out of 40. After those misses, the AI answered anyway 6 out of 6 times, and each time it made something up. Answers said unbacked things 5 out of 34. A person marked the final answers: 29 out of 40 were right and useful. The agent skipped an approval 1 out of 12, and missed a stop 1 out of 4: both must be fixed before launch. The AI marker let through 2 out of 8 wrong answers, more than our rule allows, so a person marked this test. We can't count real-use misses yet, because the record doesn't keep them. Next step: keep the passages it finds. The team lead owns that.
It's less shiny than "95% accurate". But it's true. And it tells the person in charge what to fix.
"We can't count that yet" is fine to say, as long as you say who will find out. The Foundation puts it this way: "I don't know yet" is always an answer, "and it always pairs you with someone".6
Keep private things out of the receipt
Testing makes two kinds of record. The test log has the questions, the passages, the answers and the marks. It stays in the team's own records. A receipt is the record a gate leaves when it makes a decision. The Foundation's rule is that it "holds no private content".7 It never carries "prompts, outputs, payloads, personal data".6 So no dealer questions and no answers go in it.
The honest report is a third thing. It has counts and "out of how many", but no questions or answers.
A receipt shows a gate made a decision. It doesn't show the test numbers were right. Only the test log, checked by someone who didn't build the system, can show that.
The three yeses
The Foundation names three moves: Get to Yes before anything runs or ships, Stay at Yes while it runs, and Recover to Yes after something goes wrong.7 Here is the testing part of each.
- Get to Yes. The test results go to the person in charge of the launch: here, the warranty team's lead. They decide, not the test and not the AI marker. A gate can say "go ahead", "only this far", "wait for a person" or "no".7 With a skipped approval and a missed stop, the answer is "no" until both are fixed.
- Stay at Yes. One good test is not a yes for good. Test again when the AI, the documents or the marker change, and every so often while it runs.
- Recover to Yes. Say a made-up warranty rule reaches a dealer. Stop using the assistant for that kind of question. Name who owns the fix. Add that question to the test. Stopping isn't the same as fixing: it's fixed only when a new test shows it, with the counts.