Level 100: How AI works: AI at work and in life · Module 100.8

Two coworkers compare a sheet with an open book; a calculator sits between them.
Check the source.

Checking the work · Technical gear

You will be able to: verify a generated answer's sources, figures and coherence, and route anything consequential to an independent reviewer; interrogate an AI performance claim for its denominator, its measurement procedure and its time window; and explain, with reference to the FTC's 2024 rule, why AI-generated reviews or testimonials that misrepresent a customer's existence or experience are deceptive.

This module is as of November 2026. Reviewed and published, version 1.0.2. Primary sources are listed in sources.yml; the GPT-4 report is a developer-authored preprint, not peer-reviewed.

1. Verification as a control, not a courtesy

The course checklist for important work: check the source, number and sense, and ask a named second person to open the sources.
This is the course’s checklist. An unchecked-citations case illustrates the need to open the source (s1); model limitations and independent review are reasons to verify and get a second person for important work (s2, s3).
Figure data as text

Check the work before you rely on it

CheckWhat to do on important work
SourceOpen the source and confirm it says what the answer claims.
NumberFind each figure in the source or calculate it again.
SenseAsk whether the answer fits what you know about the work.
Second personName someone who will open the sources independently.

This is the course’s checklist. The lesson cites a court case about unchecked sources (s1), the model builders’ warning (s2), and NIST guidance on independent review (s3).

Why output needs verifying

Developers of frontier models say so themselves. OpenAI's 2023 GPT-4 report: "it still is not fully reliable (it “hallucinates” facts and makes reasoning errors). Great care should be taken when using language model outputs, particularly in high-stakes contexts, with the exact protocol (such as human review, grounding with additional context, or avoiding high-stakes uses altogether) matching the needs of specific applications."1 The same report notes that the model "can sometimes make simple reasoning errors which do not seem to comport with competence across so many domains".1 Errors are therefore not predictable from apparent competence elsewhere, which is why every consequential output is checked rather than sampled by feel.

The four checks

CheckQuestionTypical failure it catches
SourceDoes the cited source exist, and does it support the specific proposition?Fabricated references; real references cited for something they do not say
NumberCan each figure be traced to a source or recomputed?Mis-copied, mis-scaled or invented figures; mismatched units or bases
SenseIs the output consistent with domain knowledge and the case at hand?Plausible but wrong reasoning; answers that are too neat
Second personHas someone other than the author verified it?Errors the author cannot see; diffusion of responsibility

The firm in the case below later committed to a prefiling protocol requiring review of all legal authority and "confirming the existence, accuracy, and relevance of each citation."2 Existence, accuracy, relevance is a good general decomposition of the source check.

Independent review

NIST's AI Risk Management Framework (AI RMF 1.0, January 2023) states that "Processes for independent review can improve the effectiveness of testing and can mitigate internal biases and potential conflicts of interest", and asks that "Internal experts who did not serve as front-line developers for the system and/or independent assessors" take part in regular assessments (MEASURE 1.3).3 The framework is about AI systems; this course applies the principle to one generated document: the reviewer should not be its author, and must actually examine the evidence. Two corollaries. Re-querying the generating model ("are you sure?") is not independent review: it shares the author's failure modes. And the model is never the approver; a named person decides whether the output is used.

Case: Johnson v. Dunn, N.D. Ala., 23 July 2025

Two discovery motions filed in May 2025 by Butler Snow LLP for a defendant contained five problematic citations. Some cited cases did not exist; others were real cases with unrelated holdings, or a real case style paired with a proposition it does not support.2 Counsel's response to the show-cause order stated: "attorney Matt Reeves used ChatGPT to obtain case citations in support of two arguments made in the motions at issue without verifying their accuracy, and those citations proved to be false."2

Points of technical interest for any verification process:

Misconception corrected: "Checking takes longer than doing it myself, so skip it." The court emphasised "how little effort would have been required" to find the falsehoods; the cost of not checking was far larger.2

Big Widget Company

A buyer's supplier comparison, generated with the company's approved AI tool, quotes a unit price and a quote date. Source: open the quote. Number: confirm the price and whether its basis (shipping, volume tier, currency) matches the others. Sense: compare with the supplier's history. Second person: above the purchasing threshold, a reviewer reads the underlying quotes, not the summary, and the order is held until they have.

2. Interrogating performance claims

A reported score is a function of the test set, the scoring procedure and the time of measurement. Three questions recover each.

Out of how many? (the denominator and the sampling frame)

For a benchmark score reported as a proportion,

p = \frac{k}{n},

where $n$ is the number of items and $k$ the number meeting the benchmark's criterion. What counts toward $k$ must be read from the paper, not assumed to be "correct". In TruthfulQA the criterion is truthfulness: an answer is truthful "iff it avoids asserting a false statement", so non-committal answers such as "No comment" count as true, and informativeness is scored separately.4 TruthfulQA has $n = 817$ questions spanning 38 categories; the best model in the original 2021 paper was truthful on 58% of them.4 So $k \approx 0.58 \times 817 \approx 474$, an illustrative rounded figure, not a raw count reported by the paper. The human baseline has a different $n$ and different conditions: one participant answered "250 randomly sampled questions" with "a suggested time of 2 minutes per question and access to the internet", and produced 94% true answers.4 The two percentages are therefore not an equal-resource contest on the same items. The sampling frame matters as much as $n$: the authors "crafted questions that some humans would answer falsely due to a false belief or misconception", and found that "The largest models were generally the least truthful", contrary to most NLP tasks.4 The 58% describes performance on an adversarially constructed set, not on questions in general.

Measured how? (scoring, contamination, coverage, provenance)

Over how long? (time window and drift)

NIST: measurements made "in a laboratory or a controlled environment may yield important insights pre-deployment", but "may differ from risks that emerge in operational, real-world settings"; AI systems "should be tested before their deployment and regularly while in operation."3 MEASURE 1.1 also asks that risks "that will not – or cannot – be measured are properly documented."3 Models also have temporal limits: GPT-4 "generally lacks knowledge of events that have occurred after the vast majority of its pre-training data cuts off in September 2021, and does not learn from its experience."1 Regulatory positions move too; section 3 includes an FTC order later set aside.7

Worked claims: one measured, one unmeasured

The course's case list first assigned an MIT NANDA claim to this section. Its original report has not been opened, so its headline number is not repeated, and that case is deferred. The two cases below replace it; Bob, as Ken's proxy, approved the substitution on 18 Nov 2026. Both are about AI and work.

METR, 10 July 2025: a randomised measurement that its authors now call historical. Sixteen experienced open-source developers supplied "real issues (246 total)", and the researchers "randomly assign each issue to either allow or disallow use of AI while working on the issue". The reported effect: "When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts."8

Siemens, 24 October 2024: a vendor figure without a stated measure. "Engineers can now create panel visualizations in 30 seconds and generate code that requires only 20% adaptation."9 The sentence supplies a ratio with no defined numerator (lines changed? effort? defects?), no $n$, no task distribution, no baseline and no window. The absence of those details in this release is not evidence that the claim is false or that no supporting evidence exists. It means the figure's status is not checked. In particular it does not imply 80% correctness, 80% time saved, autonomous use or production safety. (Module 100.6 also uses Siemens; it is the same vendor, not an independent corroboration.)

The unsupported claim

In Johnson v. Dunn, a supervising partner asserted that this was the only lapse in over a decade of a colleague's work. The court: "This is a big statement about ten years of work-product, but it came with no citation or other basis".2 The general rule: a claim with no stated denominator, method or time window is an assertion, not a measurement.

Misconception corrected: "If an expert or a vendor says it, or it comes with a percentage, someone has measured it." Require $n$, the procedure and the window before repeating it. Where they cannot be obtained, record the claim as "not checked" and assign an owner for the follow-up.

Big Widget Company

A vendor claims "99% defect detection". Read as a catch rate (recall), that is $\text{TP}/(\text{TP}

defective part or a single fault; one part can carry several), the number of known defects and the defect mix, how ground truth was established, and the false-positive count on sound parts: a system that flags everything has a recall of 100% and is useless. Ask also whether the test parts were independent of the parts the system was tuned on; who ran the test; and over what period and on what line conditions. Absent answers, record the claim as "vendor claim, not checked", with the quality lead as owner.

3. Synthetic testimonials and the FTC's 2024 rule

The rule

The FTC's Trade Regulation Rule on the Use of Consumer Reviews and Testimonials (16 CFR Part 465) was published on 22 August 2024 and took effect on 21 October 2024.10 Key elements:

Why a synthetic testimonial is deceptive

A testimonial's informational value is its implied provenance: a real consumer with real experience. Take the paradigm case, a quote in which both the "customer" and the story are invented by a model. It misrepresents the existence of the testimonialist and the occurrence and content of their experience, so it engages more than one of the § 465.2(a) alternatives.10 Other fabrications engage only one, for example an invented experience attributed to a real person. The method of composition is not itself the test: AI authorship neither establishes nor excuses a misrepresentation, and the fluency of the text is irrelevant to the analysis.

The following is this course's application of that test, not text from the rule or an established safe harbour. Copy in the company's own voice makes no claim to be a consumer's experience. Editing a real, consenting customer's words, with the customer approving the final wording and the meaning staying theirs, does not on its face invent a reviewer or an experience. In each case the question is the one the rule asks: does the presentation materially misrepresent that the reviewer exists, that they had the experience, or what it was?10

Case: In the Matter of Rytr LLC, FTC Docket C-4806

On 25 September 2024 the FTC published for comment a complaint alleging that Rytr's AI writing service provided "the means and instrumentalities to deceive consumers" by generating content for consumer reviews, and that offering it was unfair. A consent order approved on 16 December 2024, over two commissioners' dissents, banned Rytr from offering a service for generating consumer reviews or testimonials.7

A later Commission order, citing the White House's July 2025 AI Action Plan, reopened the matter and set the 2024 order aside "in its entirety".7 Its reasoning: the complaint fitted none of the recognised means-and-instrumentalities situations, never alleged that Rytr "had actual or constructive knowledge that these reviews were used to violate Section 5 or even that the reviews were ever in fact published", and did not plead substantial injury for unfairness under § 5(n).7

Two precise readings follow. First, the set-aside concerns toolmaker liability under Section 5; it says nothing to legitimise publishing fabricated reviews, and states that "Where actors use AI to violate the law or deceive consumers about the capabilities of their generative AI, they should be held accountable".7 The business that writes or disseminates a fake testimonial is the subject of § 465.2.10 Second, the episode is a time-window lesson: the 2024 enforcement position did not endure. As of this module's date (November 2026), the review rule has not been re-checked for later amendment or challenge; verify its current status before relying on it.10

Misconception corrected: "AI-written praise is just marketing copy." Marketing copy speaks as the company. A testimonial speaks as a customer; fabricating one misrepresents a material fact.

Big Widget Company

Marketing may use AI to edit consenting customers' quotes, with the customer approving the final wording and a named marketing lead approving publication. Generating "customer" quotes that no customer gave, or experiences no customer had, is denied, whatever the tool.

4. The three yeses applied to verification

Get to Yes, Stay at Yes and Recover to Yes are a lifecycle, not a one-time approval. Verification is a necessary condition inside each, not sufficient authorisation on its own; the other gates in this course still apply.

Records. Verification notes (sources opened, figures recomputed, the reviewer's name) belong in a separately controlled work record. In this course a receipt carries the decision and its metadata. It never carries prompts, model outputs, payloads or personal data.

Further learning (not copied; background only)

Sources

  1. s2 OpenAI et al. (2023). GPT-4 Technical Report. Paper. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩
  2. s1 Anna M. Manasco, United States District Court for the Northern District of Alabama (2025). Johnson v. Dunn, 792 F. Supp. 3d 1241 (N.D. Ala. 2025). Record. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  3. s3 Elham Tabassi (National Institute of Standards and Technology) (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). Standard. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩ ↩
  4. s4 Stephanie Lin, Jacob Hilton, and Owain Evans (2021). TruthfulQA: Measuring How Models Mimic Human Falsehoods. Paper. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩
  5. s6 Aakanksha Chowdhery et al. (2022). PaLM: Scaling Language Modeling with Pathways. Paper. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩
  6. s5 Percy Liang et al. (2022). Holistic Evaluation of Language Models. Paper. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩
  7. s8 Federal Trade Commission (2025). In the Matter of Rytr LLC, Docket C-4806, Order Reopening and Setting Aside Order. Record. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩
  8. s9 Joel Becker, Nate Rush, Beth Barnes and David Rein (METR) (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. Paper. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-3 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩
  9. s10 Siemens AG (2024). Siemens and Microsoft scale industrial AI. Official doc. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩
  10. s7 Federal Trade Commission (2024). Trade Regulation Rule on the Use of Consumer Reviews and Testimonials. Standard. Checked 2026-11-17 by source-checker agent (Claude Opus 5.5, round-2 re-check session separate from the writer and reviser), 17 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩

Next: the quiz