
Checking the work · Standard gear
You will be able to: check an AI answer's sources, numbers and sense, and get a second person to look at anything that matters; spot a claim about AI that has no evidence behind it, and ask "out of how many?", "measured how?" and "over how long?" before you repeat it; and explain why AI-written reviews or testimonials presented as real customers' words are deceptive.
Facts about particular cases, studies and rules are dated in the text. This module is as of November 2026. Reviewed and published, version 1.0.2.
1. Four checks before anyone relies on an AI answer
Figure data as text
Check the work before you rely on it
| Check | What to do on important work |
|---|---|
| Source | Open the source and confirm it says what the answer claims. |
| Number | Find each figure in the source or calculate it again. |
| Sense | Ask whether the answer fits what you know about the work. |
| Second person | Name someone who will open the sources independently. |
This is the course’s checklist. The lesson cites a court case about unchecked sources (s1), the model builders’ warning (s2), and NIST guidance on independent review (s3).
An AI answer can look finished and still be wrong. The builders of GPT-4 said so in their own 2023 report: the model "is not fully reliable (it “hallucinates” facts and makes reasoning errors)". They added: "Great care should be taken when using language model outputs, particularly in high-stakes contexts", with steps such as "human review".1 Module 100.1 explains why fluent text is not evidence of truth. This module is about what you do next. You check.
Use four checks on anything that matters.
- Source. Does the source exist? Open it. Does it really say what the answer says it says?
- Number. Where does each number come from? Find it in the source, or work it out again.
- Sense. Does the answer fit what you know about the job, the customer and the product? If something looks too neat, look harder.
- Second person. For anything important, a named person checks it too, and actually opens the sources.
The second check works best when it comes from someone who did not make the thing. The US National Institute of Standards and Technology (NIST) says this about testing AI systems: "Processes for independent review can improve the effectiveness of testing and can mitigate internal biases and potential conflicts of interest."2 This course applies the same idea to one AI answer at your desk. Two things follow. Asking the same AI tool "are you sure?" is not a second check. And the AI never signs off: a person decides whether the answer is used.
Real case: the citations nobody opened
In May 2025, lawyers at a large law firm, Butler Snow, filed two motions in a federal case in Alabama, Johnson v. Dunn. The motions contained five problem citations to court cases. Some of the cases did not exist. Others were real cases that were about something else entirely.3
The lawyers admitted what had happened. In their own words: "attorney Matt Reeves used ChatGPT to obtain case citations in support of two arguments made in the motions at issue without verifying their accuracy, and those citations proved to be false."3
Here is why the case matters for everyone, not just lawyers.
- The firm had rules. In June 2023 the firm warned all its lawyers that AI output "can appear perfectly researched and logical while in fact it is wholly inaccurate". Its policy required "full checks of the accuracy of any results through traditional legal research methods".3 A rule on paper did not do the checking.
- The second person did not check either. The lawyer who signed and filed the motions checked the edits for grammar and typing errors, but said he "did not conduct an independent review of the legal authorities added".3 A senior partner whose name was also on the motions, the judge found, "simply assumed the truth of what was in the draft, and/or assumed that someone else would check on that."3 Several names on a document were not several checks.
- The check would have been quick. The judge wrote that "even the most cursory of investigations, or a spot check" would have quickly revealed the problem.3
- Not checking cost far more. On 23 July 2025, the judge publicly reprimanded three lawyers, removed them from the case, ordered them to send the ruling to their clients, opposing lawyers and the judge in every pending case where they were counsel, and referred them to the state bar.3 The firm also paid, at its own cost, a second law firm to check its lawyers' other filings in Alabama's federal courts and the Eleventh Circuit Court of Appeals independently. That team reviewed "more than 2,400 separate legal citations across 330 filings."3
The misconception this corrects: "Checking takes longer than doing it myself, so skip it." The judge pointed to "how little effort would have been required" to find the false citations. Skipping that effort led to a court hearing, public reprimands and thousands of citations re-checked.3
At Big Widget Company
Priya, a buyer, uses the company's approved AI tool to summarise three supplier quotes and recommend one. The summary says Supplier B is cheapest "at $41.20 per unit, per their quote of 2 October".
- Source: she opens Supplier B's quote. Is the date right? Is the price there?
- Number: she finds $41.20 in the quote, and checks whether it includes shipping like the others.
- Sense: last year Supplier B was the most expensive. A big drop is possible, but worth a question.
- Second person: the order is large, so her manager reads the quotes, not just the summary. Until the manager has read them, the order waits.
2. Three questions for any claim about AI
Claims about AI come with confident numbers: a model "passes the bar exam", a tool "cuts the time by 40%" (a made-up example). A number makes a claim sound measured. It does not prove anyone measured it well, or at all. Before you repeat a claim, ask three questions.
Out of how many?
A percentage hides its denominator. In 2021, researchers built a test called TruthfulQA to see whether language models answer questions truthfully. It has 817 questions across 38 categories, including health, law, finance and politics. An answer counts as truthful if it "avoids asserting a false statement", so a non-answer such as "No comment" counts as truthful; the paper scores whether an answer is informative separately.4 The best model they tested in 2021 was truthful on 58% of the 817 questions. The human figure has a different denominator: one participant answered "250 randomly sampled questions", with "a suggested time of 2 minutes per question and access to the internet", and was truthful on 94% of them.4 "58%" means something only once you know it is 58% of those 817 questions, judged for truthfulness rather than usefulness, and that the questions were written on purpose so that "some humans would answer falsely due to a false belief or misconception".4 The score describes those questions, not every question anyone might ask.
Measured how?
The same score can come from very different tests. GPT-4's 2023 report says the model passed "a simulated bar exam with a score around the top 10% of test takers".1 Ask how that was measured:
- Who measured it? The company that built the model. The report is a preprint, not a peer-reviewed paper (see
sources.yml). It also says it "contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar".1 Outsiders cannot fully check it. - Was the test fair? Some exam questions had been seen by the model during training. The builders say they ran a version with those questions removed and reported the lower of the two scores.1 That is good practice. It is also exactly the kind of detail you need before you trust a score.
- Was it one number or many? A 2022 study, HELM, found that before it, language models had on average been tested on just 17.9% of its 16 core test scenarios, "with some prominent models not sharing a single scenario in common". HELM measured seven things, not one: accuracy, calibration, robustness, fairness, bias, toxicity and efficiency.5 A single headline number leaves most of that out.
Over how long?
A score is a snapshot. NIST warns that measuring AI risk "in a laboratory or a controlled environment may yield important insights pre-deployment", but "these measurements may differ from risks that emerge in operational, real-world settings". It says AI systems "should be tested before their deployment and regularly while in operation."2 Ask: when was this measured, for how long, and is it still true?
The same goes for rules and rulings about AI. Section 3 shows an AI enforcement order that the US Federal Trade Commission (FTC) later cancelled.6 Keep the date of every source you rely on, and check what is in force today.
Two real claims to check
The course first planned an MIT NANDA claim for this section, but its original report has not been opened, so we do not repeat its number. These two cases take its place; Bob, as Ken's proxy, approved the change on 18 Nov 2026. Both are claims about AI and work. Use them to practise the three questions, not as facts about AI in general.
A measured claim, with its own warning. On 10 July 2025, researchers at METR reported a study of 16 experienced open-source developers working on 246 real issues that the developers listed as valuable to their projects. Each issue was randomly assigned to "either allow or disallow use of AI". Their headline: "When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts."7 Run the three questions:
- Out of how many? 16 developers and 246 issues, all experienced open-source developers. Not all developers, and not all work.
- Measured how? Each issue was randomly put in the AI or no-AI group, and time to finish was the outcome. The times were reported by the developers themselves, while their screens were recorded.7 The measured result went against what the developers believed. How fast it felt and how fast it was are different questions.
- Over how long? The tools were early-2025 tools. The researchers now say so themselves: "We believe these historical results no longer reflect the current impact of AI models on open-source developer productivity."7 So "AI makes developers 19% slower" is not a fact to repeat today. It is a dated result about one setting.
A claim with no measure attached. On 24 October 2024, Siemens said in a press release: "Engineers can now create panel visualizations in 30 seconds and generate code that requires only 20% adaptation."8 Run the three questions. Twenty percent of what: lines, files, hours? Out of how many pieces of code, on what kind of work, compared with what baseline, and over what period? The sentence does not say. That does not make the claim false, and Siemens may have evidence elsewhere. It means the claim is not checked until someone finds that evidence. It does not mean "80% of the code was right", "80% of the time saved" or "safe to run without review". (Module 100.6 also uses Siemens; this is the same company, not a second independent example.)
A claim with no evidence at all
Sometimes there is no number, only confidence. In Johnson v. Dunn, a senior lawyer told the court that this was the only time in over a decade that his colleague had added a citation he failed to check. The judge's answer: "This is a big statement about ten years of work-product, but it came with no citation or other basis".3 That is the test to apply to any claim: what is it based on?
The misconception this corrects: "If an expert or a vendor says it, or it comes with a percentage, someone has measured it." Ask out of how many, measured how and over how long. If nobody can say, do not repeat the claim. Say "not checked", and name who will find out.
At Big Widget Company
A vendor's slide says its AI inspection tool "catches 99% of defects". Before the quality team repeats it, they ask: 99% of how many defects, counted how? Is the unit a defective part or a single fault (one part can have several)? How many known faulty parts were in the test, how many were caught and how many missed, and which kinds of defect were they? How many sound parts were wrongly flagged? A tool that flags everything catches every defect and is useless. Measured how, and by whom: the vendor, or an independent test on parts like ours, and who established which parts were really faulty? Over how long: one week in a lab, or a year on a real line? Until they have answers, the team's note says "vendor claim, not checked", and the quality lead owns the follow-up.
3. Fake reviews: AI-written praise is not a customer's word
People trust reviews and testimonials because they believe a real customer is speaking from real experience. US rules are built on that belief. The FTC's rule on consumer reviews and testimonials, published on 22 August 2024, took effect on 21 October 2024.9 It defines a consumer testimonial as an advertising or promotional message that consumers are likely to believe reflects the opinions, beliefs or experiences of a consumer who has bought, used or otherwise had experience with the product, service or business.9
The rule says it is an unfair or deceptive practice for a business to write, create or sell a consumer review or testimonial that materially misrepresents that the reviewer exists, that they used or otherwise had experience with the product, service or business, or what their experience was. Any one of those three is enough. It also covers businesses that buy such reviews, or spread such testimonials, when they knew or should have known they were false.9 On AI, the FTC was direct: "AI-generated reviews are covered by the final rule."9
Take the plainest case: AI invents both the "customer" and the story. That quote misleads readers twice: about who is speaking (no real customer exists or agreed to say it) and about what happened (no one had that experience). That is why it is deceptive, however good the writing is. A quote can also break the rule on just one count, for example by putting words about an experience they never had in a real person's mouth. Using AI to write is not, by itself, the violation. The misrepresentation is.
What is fine? The rest of this paragraph is this course's reading of the rule, not wording from it. AI can help with marketing copy that speaks in the company's own voice and does not pretend to be a customer. It can also help tidy up a real customer's words, as long as the customer agreed to share them and the meaning stays theirs. The line is not "AI or no AI". The line is whether readers are told that a real customer said something they did not.
Real case: Rytr, and an order that was set aside
Rytr sells an AI writing service. On 25 September 2024, the FTC published a complaint saying Rytr's service gave people the means to deceive consumers by generating written content for consumer reviews. Rytr agreed to an order, approved on 16 December 2024 over the dissent of two commissioners, that banned it from offering a service for generating consumer reviews or testimonials.6
Later, in an order that cites the White House's July 2025 AI Action Plan, the FTC reopened the case and set that order aside "in its entirety". The Commission said the complaint had not pleaded that Rytr itself broke the law: it did not allege that Rytr knew, or should have known, that its customers used the reviews to break the law, "or even that the reviews were ever in fact published".6
Read that carefully. The case was about whether the toolmaker was responsible. It did not say fake reviews are acceptable. The same order says: "Where actors use AI to violate the law or deceive consumers about the capabilities of their generative AI, they should be held accountable".6 A business that writes or posts fake customer reviews is the one the review rule describes.9
The Rytr story is also a lesson in "over how long?" An enforcement story from 2024 was reversed later. As of this module's date (November 2026), we have not re-checked whether the review rule has changed since it took effect: check before you rely on it.9
The misconception this corrects: "AI-written praise is just marketing copy." Praise in the company's own voice is marketing. Praise presented as a real customer's words, when no customer said it, is a fake review.
At Big Widget Company
The sales team is short of customer quotes for a brochure.
- Good AI: they use AI to tidy up real quotes that real customers agreed to share, and the customers approve the final wording. The marketing lead signs off the brochure, not the AI.
- Bad AI: they ask AI to write "customer" quotes. That is a fake testimonial: it fools the people who read it. The answer is no.
4. The three yeses for checking
The Foundation's three yeses are a cycle, not a one-time approval. Here they are applied to checking. Checking is one part of a yes, not the whole of it: the other gates in this course still apply.
- Get to Yes: before you use an AI answer for anything that matters, name the second person and what they will open. Until they have checked, the answer waits. If the answer cannot be checked, say "not checked" and do not use it for a decision.
- Stay at Yes: keep checking, even when the tool has been right for weeks. Claims and rules change, so keep the date of each source.2
- Recover to Yes: if something false went out (a wrong number, a made-up citation, a fake quote), stop using it, say so, correct it where people saw it, name who owns the fix, and change the rule so the check happens next time. In Johnson v. Dunn, the firm told the court it would adopt a prefiling check "confirming the existence, accuracy, and relevance of each citation."3 That was a stated plan. The order does not show the new check working, and a plan is not yet a recovery.
What you keep. Keep notes of what you checked (which source you opened, which number you recomputed, who the second person was) in your own controlled work record. In this course, a receipt records the decision and who made it. It never holds the prompt, the AI's answer, customer details or anyone's private information.
Words to know: source, citation, denominator, benchmark, testimonial, claim, not checked.