Level 200: Using gen AI well · Module 200.2

Three coworkers lean over a sheet as one points with a pencil.
Show what I can check

Prompts that hold up · Technical gear

Objectives and scope

Shown work may help but can contain errors or misstate how the answer was reached. Use a calculator for sums, open sources for quotes and citations, and check the conclusion.
Shown steps can help and still contain mistakes (s6); they may also misrepresent how the model arrived at the answer (s7). The verification checklist is course advice.
Figure data as text

Treat each step as a claim

PartWhat it means
WHAT IT TELLS YOU [s6, s7]Something to check, not proof. Shown steps can help, but can be wrong. They may not explain how it got the answer.
COURSE CHECKLISTUse an independent check. Redo sums with a calculator or spreadsheet. Open the source for quotes and citations. Check the final total or conclusion too.

Lesson 100.5 treated a prompt as a one-off specification. Here the unit of analysis is a template applied to a stream of inputs, and the question is how it behaves across that stream, not on one draw. The evidence is four peer-reviewed papers (NeurIPS 2020, NeurIPS 2022, NeurIPS 2023, ICLR 2024), 1234 one field study published in The Quarterly Journal of Economics,5 and one vendor's product documentation, read on 18 November 2026.6789 Vendor guidance and the course's own inferences are labelled as such. The governance vocabulary (gates, the three yeses, receipts) is the AiGovOps Foundation's.1011

Scope boundary: the model drafts and a named person decides. Instructions in a template condition output; they are not an enforced control (lesson 200.8). The Foundation states the boundary as "No direct model-to-tool path for consequential actions."10 A template's output is a draft for a person, not an action. Building a scored test set with a rubric, and the limits of model-graded evaluation, are lesson 200.4; this lesson stops at a pre-deployment smoke test of one template.

A template as a function over an input distribution

The following framing is the course's, not taken from one paper. Let a template $T$ be a fixed string with named slots, and $x$ the per-case values (a customer email, a claim record). The prompt is $T(x)$, and the tool returns a sample

y \sim p_\theta\big(\,\cdot \mid T(x)\big).

Let $s(x, y) \in \{0, 1\}$ be a written acceptance criterion: 1 if $y$ is right and usable for input $x$. What matters for a recurring task is the template's acceptance rate over the inputs it will actually see, drawn from a distribution $\mathcal{D}$:

\operatorname{acc}(T) = \mathbb{E}_{x \sim \mathcal{D},\; y \sim p_\theta(\cdot \mid T(x))}\big[s(x, y)\big].

One trial on one easy $x$ is a single sample from that expectation, taken from the part of $\mathcal{D}$ least likely to fail. 200.2-a is about choosing $T$; 200.2-c is about estimating $\operatorname{acc}(T)$ honestly, including the tails of $\mathcal{D}$.

200.2-a: the six fields of a template

The course's six fields, with the evidence for each:

FieldRole in $T$Evidence
JobTask specification; names the human who checks and sendsTask descriptions improve few-shot performance1
AudienceRegister, length, reading levelVendor: be specific about format and constraints7
FactsTyped slots for $x$, plus "use only these"Vendor: tag "variable inputs" apart from instructions7
Example$K$ demonstrationsIn-context demonstrations, no gradient updates1
Output formatFixed schema, including a provenance list per claimVendor: be specific about output format7
When unsureAn explicit abstention path with a fixed sentinelVendor: permission to admit uncertainty9

Demonstrations. Brown et al. evaluated GPT-3 "without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model", in zero-, one- and few-shot settings.1 Performance improved with a natural-language task description and with $K$, but larger $K$ was "usually but not always better".1 Anthropic's guidance for its own models (as of November 2026) says demonstrations should be "Relevant: Mirror your actual use case closely" and "Diverse: Cover edge cases and vary enough that Claude doesn't pick up unintended patterns", and suggests three to five.7 Neither source tells you the right $K$ for your task; that is an empirical question for 200.2-c.

Delimiting slots. Anthropic recommends XML-style tags because they "help Claude parse complex prompts unambiguously, especially when your prompt mixes instructions, context, examples, and variable inputs", with "consistent, descriptive tag names across your prompts".7 Tag names used in this module, such as <claim_record>, are the course's own examples, not the vendor's. Vendor-specific guidance; the general engineering point is separation of the fixed instruction text from per-case data, so that template edits and data substitutions are distinct, reviewable changes. Delimiters are a formatting aid, not a security boundary; what text inside a slot can do to a connected assistant is lesson 200.5.

Output schema with provenance. A fixed schema turns checking into a per-field comparison. The course's pattern is a trailing list of every date, amount and commitment in $y$, each mapped to the slot line it came from. This is a checking aid, not a guarantee: the list is itself model output and can be incomplete, so the checker compares against the slot data, not only against the list.

Abstention. Anthropic's guidance is to "explicitly give Claude permission to admit uncertainty", which it says "can drastically reduce false information"; the same page says its techniques "don't eliminate them entirely" and to "always validate critical information".9 Both are vendor claims about its own product. Making the abstention a fixed sentinel (e.g. NEEDS A PERSON:) has a testing benefit: abstention becomes a countable outcome, and for some test inputs it is the correct outcome. It is also a routing decision, not just a refusal: the sentinel is the template's "wait for a person" answer, and the job field should name who picks the case up. The Foundation's phrasing: "'I don't know yet' is always an answer, and it always pairs you with someone."11

Anchor case: capturing tacit practice

Brynjolfsson, Li and Raymond studied a generative-AI assistant deployed to technical-support agents at a Fortune 500 business-process software firm, rolled out primarily in fall 2020 and winter 2021.5 The system was built on GPT-3 and fine-tuned on customer-agent conversations labelled with outcomes such as resolution and handling time, with top performers' chats up-weighted, followed by further training "similar in spirit to Ouyang et al. (2022)".5 The authors note that "many aspects of successful agent behavior are difficult to quantify" and that the training helped the model distinguish top performers' successful behaviours, "including those they tacitly apply".5 Outputs were suggested replies and links to internal documentation, shown only to the agent, who had "full discretion over which, if any, AI suggestions to accept".5

Identification is a difference-in-differences design on the staggered rollout, $y_{it} = \delta_t + \alpha_i + \beta\,\mathrm{AI}{it} + \gamma X{it} + \epsilon_{it}$.5 Results in the published article: a 15% increase in chats resolved per hour; a 30% increase in issues resolved per hour for less skilled and less experienced workers; little effect on higher-skilled or more experienced workers, with a small decrease in conversation quality for the most skilled; gains largest for relatively rare problems; and no suggestions where training data on a topic was insufficient.5 The authors frame these as "medium-run effects in a single firm".5 (The 2023 working paper reports different sample sizes and estimates; this module uses the published article throughout.)

Two cautions on transfer. First, this is supervised fine-tuning on outcome-labelled data, not a prompt template. The study tested no written prompt, so it is no evidence that a template transfers practice. The course's inference, not the authors', is weaker: if some tacit practice could be learned from examples, it is reasonable to write down the parts of practice that can be stated. Whether a given template helps is not checked until it is tested (200.2-c). Second, the heterogeneous effects (by worker skill, by problem frequency) are exactly the kind of variation a single trial cannot reveal, which is the argument of 200.2-c.

200.2-b: decomposition, elicited working, and faithfulness

Elicited reasoning can raise accuracy. Wei et al. showed that few-shot exemplars containing intermediate reasoning steps improve performance "on a range of arithmetic, commonsense, and symbolic reasoning tasks" for sufficiently large models; a 540B-parameter model with eight such exemplars reached state-of-the-art accuracy on GSM8K, "surpassing even finetuned GPT-3 with a verifier".2

The chain is not guaranteed to be correct. The authors state that "there is no guarantee of correct reasoning paths, which can lead to both correct and incorrect answers", and that the work "does not answer whether the neural network is actually 'reasoning'".2 In a manual analysis of 50 incorrect GSM8K outputs from LaMDA 137B, 46% of chains were almost correct barring minor mistakes and 54% had major semantic or coherence errors; 8% were completely correct except for a calculator error.2 A post-hoc external calculator (Python eval on the arithmetic expressions) "significantly boosts performance of chain-of-thought prompting on most tasks".2 The calculator was applied to the arithmetic benchmarks only, so "most tasks" here means most of those.2 The practical reading: arithmetic inside a chain is a known failure point, and an external tool for it helps.

The chain is not guaranteed to be faithful. Turpin et al. added biasing features to inputs, for example reordering multiple-choice options in a few-shot prompt so the answer is always "(A)".3 The features were not concealed: they sat in the prompt, visible to any reader. The models' explanations did not mention these features, and accuracy dropped by as much as 36% across 13 BIG-Bench Hard tasks with GPT-3.5 and Claude 1.0.3 Their conclusion: chain-of-thought explanations "can systematically misrepresent the true reason for a model's prediction".3

So treat elicited working $z$ as a set of checkable claims, not as a trace of the computation that produced $y$. Verifying each claim in $z$ catches errors that are in $z$; it does not establish that $y$ follows from $z$ for the reasons stated. Hence the course's protocol:

  1. Decompose explicitly. Specify the steps in $T$ (extract inputs → compute → draft), one output field per step.
  2. Require independently checkable working. Each number with its source location, each quote verbatim with its location, each computation as an explicit expression.
  3. Verify each step externally. Quotes against the source document; arithmetic recomputed with a calculator or spreadsheet, not by asking the same model.2 (Model self-feedback is not verification; lesson 100.5.)
  4. Verify the conclusion independently. Recompute the final figure from the checked inputs and re-read the final text, because a fully checked chain need not be the cause of the answer.3

Worked example (made up). Expected annual warranty cost $C = N \cdot r \cdot c$ with $N = 1{,}240$ units, claim rate $r = 0.035$, mean repair cost $c = \$820$. The model's working reads: $N r = 434$; $434 \times 820 = \$355{,}880$. Step 3 is internally consistent, which is what makes the error easy to miss. External recomputation gives $N r = 43.4$ and $C = 43.4 \times 820 = \$35{,}588$: the middle step is off by a factor of ten. A consistent chain from a wrong intermediate is the case Wei et al.'s "calculator error" category describes.2

200.2-c: testing a template before relying on it

Sensitivity to non-semantic variation. Sclar et al. found that "several widely used open-source LLMs are extremely sensitive to subtle changes in prompt formatting" in few-shot settings, with "performance differences of up to 76 accuracy points when evaluated using LLaMA-2-13B"; the sensitivity persisted with larger models, more few-shot examples or instruction tuning.4 They recommend reporting a range of performance across plausible formats rather than a single number.4 The implication for a template: an edit that looks cosmetic is a change to $T$ and needs re-testing.

Vendor guidance converges. Anthropic's overview assumes "a clear definition of the success criteria for your use case" and "some ways to empirically test against those criteria" before prompt iteration begins.6 Its evaluation guide says success criteria should be specific, measurable, achievable and relevant, that evals should "mirror your real-world task distribution", and "Don't forget to factor in edge cases!", listing irrelevant or nonexistent input, overly long input, poor or harmful user input, and ambiguous cases where humans would disagree; its sample tests label typos, multi-topic inputs and an angry customer as edge cases.8

The course's smoke test.

  1. Write $s(x, y)$ before collecting outputs: what $y$ must contain, what it must never do, and when abstention is the correct output.
  2. Assemble $n \approx 6$–$10$ inputs that cover the head of $\mathcal{D}$ and its known tails: a routine case, a no-op (thank-you note), an adversarial or hostile input, an input the slot facts cannot answer (correct output: abstain), a multi-topic input, a long input with the key question buried, noisy input, and an input requesting a forbidden commitment.
  3. Data rule: synthetic inputs, or real inputs approved for this use and redacted. Never live customer or company records (lessons 100.7, 200.5).
  4. Record the expected outcome for each input before running it, to avoid post-hoc acceptance.
  5. Run each input; re-run the tail cases more than once, since sampled outputs vary (lesson 200.1).
  6. After any change to $T$, the tool or the model version, re-run the full set (regression testing).

What a small pass tells you. This step is the course's arithmetic, not a cited result. If all $n$ independent trials pass, the exact one-sided 95% upper bound on the failure rate $f$ solves $(1 - f)^n = 0.05$:

f_{\max} = 1 - 0.05^{1/n}, \qquad n = 8 \;\Rightarrow\; f_{\max} \approx 0.31.

That bound assumes the $n$ inputs are independent random draws from one fixed input population. The smoke test above is not that: its inputs are hand-picked tail cases, and some are re-run more than once. So even the loose bound does not apply to $\operatorname{acc}(T)$ on real traffic. A curated smoke set gives no statistical bound on $\operatorname{acc}(T)$ at all. The arithmetic shows only that, even under the best-case assumption, eight clean passes are consistent with a template that fails on nearly a third of inputs. Report the count ("8 of 8 curated cases passed"), not a rate. A smoke test can show a template is not ready; it cannot show that it is. Building a larger scored test set with a rubric, and what a passing rate on it means, is lesson 200.4.

The three yeses, mapped to the template lifecycle. The Foundation names three movements, Get to Yes, Stay at Yes and Recover to Yes, timed "before a single action runs", "while it runs, in real time" and "after — on demand, on incident"; its gate returns one of "go ahead · only this far · wait for a person · no".10 The mapping below is the course's application, not a quotation.

MovementFor a templateGate answer
Get to Yes$s(x, y)$ written, smoke set run, a named owner approves the template for a stated scope"only this far" (scope-limited go ahead); "wait for a person" until the set has run
Stay at YesEvery draft checked by a person before sending; regression set re-run on any change to $T$, tool or model versionAn approval is conditional on the regression set staying green
Recover to YesOn a bad draft reaching a customer: stop use, name the owner, correct what was sent, add the failing input to the set, fix $T$, re-run the full setA fix without a re-run is a plan, not a recovery

Evidence, not content. The test set, expected outcomes and marks are a work record, kept where the workplace allows. They are not a receipt. A Foundation gate "leaves a signed receipt that holds no private content",10 and the receipt never carries "prompts, outputs, payloads, personal data".11 Template text, test inputs and drafts stay in the work record.

Applied: a Big Widget dealer-reply template (made up)

Big Widget Company and its people are invented for this course.

Sam's dealer-support team replaces an ad-hoc prompt ("Write a friendly reply to this dealer email"), validated on one thank-you note, with a six-field template adapted from the customer-service one in 200.2-a. The fields stay; their values change for dealers. The job names the agent who decides whether to send. The audience becomes "a dealer's service desk; they know the products; short and exact, with part numbers". The slots are the warranty terms, claim record and dealer email, and the demonstration is a synthetic dealer reply. The template also has a schema ending in a provenance list, and a NEEDS A PERSON: abstention path. Where a reply involves a computed amount, the template requires each input with its source line and each computation on its own line, and the agent recomputes it externally. Before rollout Sam writes $s(x, y)$, builds eight synthetic dealer emails covering the tails above, and records expected outcomes. Run 1 fails both complaint inputs (an unlisted call-back commitment); Sam revises the abstention field and re-runs all eight, which pass. Sam, as the named owner, approves the template for dealer warranty emails only ("only this far"). The set is kept as a regression suite, re-run on every change to the template or the tool. Every draft is still checked by an agent against the claim record before sending, and the team treats eight passes as "not shown broken on eight curated cases", not as a measured accuracy.

Summary

Further learning (optional, not citations)

Free public university material. It is not a source for anything taught above.

Sources

  1. s1 Tom B. Brown et al. (2020). Language Models are Few-Shot Learners. Paper. Checked 2026-11-18 by source-checker (separate Claude Code session 9089430c). ↩ ↩ ↩ ↩ ↩ ↩
  2. s6 Jason Wei et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Paper. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  3. s7 Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. Paper. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩ ↩ ↩ ↩
  4. s8 Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr (2023). Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. Paper. Checked 2026-11-18 by source-checker (separate Claude Code session 9089430c). ↩ ↩ ↩ ↩
  5. s9 Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond (2025). Generative AI at Work. Paper. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  6. s2 Anthropic (2026). Prompt engineering overview (Claude documentation). Official doc. Checked 2026-11-18 by source-checker (separate Claude Code session 9089430c). ↩ ↩ ↩
  7. s3 Anthropic (2026). Prompting best practices (Claude documentation). Official doc. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩ ↩ ↩ ↩ ↩
  8. s4 Anthropic (2026). Define success criteria and build evaluations (Claude documentation). Official doc. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩
  9. s5 Anthropic (2026). Reduce hallucinations (Claude documentation). Official doc. Checked 2026-11-18 by source-checker (separate Claude Code session 9089430c). ↩ ↩ ↩ ↩
  10. s10 AiGovOps Foundation (2026). Manifesto. Official doc. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩ ↩
  11. s11 AiGovOps Foundation (2026). AiGovOps Foundation Practice (home page). Official doc. Checked 2026-11-18 by source-checker (separate Claude Code session 1ac87f2f, round 2 re-check). ↩ ↩ ↩

Next: the quiz