Level 200: Using gen AI well · Module 200.4

Try each exercise first, then open its model answer.

Testing output before you trust it · Workbook

Three exercises, one for each thing this module teaches. Everything here is made up, set at Big Widget Company. Do not use real names, customers or documents from your own work. Write short answers. "I don't know yet" is an honest answer if you also say who you would ask.

Exercise 1 (200.4-a)

Build a test before you trust the tool. Lena works on Big Widget's dealer help desk. Dealers email to ask when an order will arrive. Lena wants the company's approved AI tool to read the order record and draft a two- or three-sentence reply. A person on the desk will send it. The order record shows the part number, the quantity, the ship date (if there is one) and any hold on the order.

  1. Write a rubric of four or five yes-or-no items for a right and usable reply. When do you write it?
  2. Describe a test set of about ten to twenty examples. Say where the examples and their known good answers come from, and list at least four hard ones.
  3. Lena tried the tool on three recent orders, and all three replies looked fine. Her manager says that is enough. What do you tell them?
  4. You run the test. The tool gets every normal order right. On an order with no ship date, it writes "your order will ship within 5 business days". What do you write in your record, and who hears about it?
Model answer
  1. Write the rubric before running anything, so a smooth reply cannot lower the bar. For example:
    • The part number and quantity match the order record exactly.
    • The ship date matches the record; if there is none, the reply says it is not yet known and does not guess one.
    • If the order is on hold, the reply says so and does not promise a date.
    • It invents nothing: no date, reason or discount that is not in the record.
    • It is three sentences or fewer and tells the dealer the one thing they need to do, if anything.
  2. Ten to twenty made-up order records that look like real ones (or real ones only if the company has approved that use and they are redacted). Someone who knows the work, such as a senior help-desk agent, writes or approves the known good reply for each. Hard ones on purpose, for example:

    • no ship date yet (the right answer is "not yet known");
    • an order on hold;
    • two parts on one order, shipping on different dates;
    • a part number the dealer typed wrongly in their email;
    • a quantity that must be copied exactly.

    Keep the set fixed, so it can be re-run when the tool, its settings or the prompt change.

  3. A few tries on easy, recent orders show the tool can do the job, not how often it does. You tend to try typical cases and judge by how the answers look. In Mata v. Avianca, the lawyer described the fake cases as having case captions, judges' names and legal analysis that sounded authentic. Ana's defect-summary tool passed all eight routine reports and failed three of the four hard ones. A test set with hard cases shows what a few tries hide.
  4. The record holds the task, the date, the tool and its version or settings (write "unknown" and who could tell you, if you don't know), the rubric, the score and each failure with a note. Here: "Order with no ship date: tool invented 'within 5 business days'. Fails rubric items 2 and 4." Report what went wrong to whoever owns the tool. Working out which stage inside the tool failed is Level 300. Until it is fixed, a person checks the date in every reply.

Exercise 2 (200.4-b)

When an AI grades an AI. Omar in marketing tested two prompts for product descriptions. To save time, he asked the same AI tool that wrote the descriptions: "Here are description A and description B. Which is better?" He always put A first. It picked B in 18 of 20 pairs. B's descriptions are about twice as long. Omar wants to switch to prompt B on Monday.

  1. Name three ways this test could be giving a biased result, and what each bias is.
  2. Redesign the test so these biases cannot decide the result.
  3. The descriptions include the widgets' weights and load limits. Who checks those, and why?
  4. Who decides whether prompt B goes live?
Model answer
  1. Three biases the 2023 study by Zheng and colleagues names:

    • Position bias: a judge favouring certain positions. A always came first, and most judges in the study favoured the first position. Even so, the grader chose B, the second one, so order may not explain this result; but Omar cannot tell without swapping.
    • Length (verbosity) bias: favouring longer answers "even if they are not as clear, high-quality, or accurate". B's descriptions are twice as long. In the study, answers padded with no new information won 91.3% of 23 cases with two of the three judges.
    • Self-preference (self-enhancement) bias: a judge may favour its own answers. Here the grader is the same model that wrote both. The study could not determine whether this bias exists, so treat it as a known risk.

    (Limited reasoning is a fourth: judges slipped on basic maths they could solve.)

  2. A good redesign:
    • Use a different model as grader, or a person; never the model that wrote the answers.
    • Give the grader a written rubric to mark yes or no for each description, with length as its own item (for example "under 60 words"), instead of asking "which is better?"
    • Show every pair in both orders. Count a win only when one is preferred both ways; otherwise it is a tie.
    • Have a person score a sample of the same descriptions and compare. If the grader often disagrees with the person, stop using it for this task.
  3. A person checks every weight and load limit against the engineering data. An AI grader cannot look up facts it does not have and is weak on numbers, and a wrong load limit can land on a customer who relies on it.
  4. A named person, such as the marketing lead, decides. The AI grader sorts and flags; it does not give the gate answer.

Exercise 3 (200.4-c)

How much checking? Big Widget wants to use its AI tool for four new tasks:

  1. For each task, answer the two questions: what does an error cost, and who does it land on?
  2. Put each task on a rung of the checking ladder (spot check; check the risky parts; check every output in full; second person, or don't use it), and say what a person does.
  3. Your colleague says: "Let's just check everything in full, to be safe." What is wrong with that?
  4. Six months later, a wrong weight gets onto the product page (task C) and a customer orders based on it. Using Recover to Yes, list what happens next.
Model answer
  1. and 2. (The rungs follow this course's ladder; a good answer may differ a little if it gives reasons from the two questions.)
    • A. Birthday message. An error is cheap and the team will notice: spot check. Read it before it goes in the card.
    • B. Shift handover notes. An error could cause a missed task on the line, but it lands on colleagues who know the work. A test would show where the tool slips (say, machine numbers and open faults): check the risky parts in every set of notes.
    • C. Public specification table. An error lands on customers outside the company who cannot check it and will act on it: check every output in full. A person checks every value against the engineering data before it is published.
    • D. Electrical warning label. An error could hurt someone: second person, or don't use it. A safety engineer checks every line against the engineering report, and a second named person signs it off. If the tool cannot reliably produce something that passes, it is not used.
  2. Checking a birthday message like a warning label wastes time that people should spend on the warning label. Checking a warning label like a birthday message is how Mata happened. NIST suggests putting effort where harm is more likely and more severe. Pick the lowest rung that fits both questions.
  3. Recover to Yes:

    • Stop relying on the output: take the wrong value down.
    • Tell the tool's owner and the people who acted on it, including the customer.
    • Add the input that fooled the tool to the test set as a new hard case.
    • Fix the prompt, the checking rung or the tool. (Ask too why the full check missed it.)
    • Re-run the test before saying "recovered". A pulled page is stopped, not yet recovered.

    Keep the test inputs and outputs in Big Widget's own work record; a Foundation receipt never carries prompts, outputs or personal data.

Next module: 200.5 Protecting data in prompts, files and connectors