Level 200 objectives

Objectives map signed: Ken Johnston, 2026-11-18 (by proxy: Ken gave his proxy to Bob Rapp, who approved and recorded it). Level 200 lessons are published.

level
200
name
Using gen AI well
for
Everyday and professional users who already use AI tools at work or at home and have finished Level 100 or placed into 200. No coding needed; the code path in the project is optional.
reading level
US grade 8, as Level 100 (Bob, 18 Nov 2026; decisions-2026-11-18.md)
setting
Big Widget Company, a made-up 100,000-person maker of complex, expensive, car-like widgets
signed by
Ken Johnston, 2026-11-18 (by proxy: Ken gave his proxy to Bob Rapp, who approved and recorded it)
modules
  • id
    200.1
    slug
    what-the-model-sees
    title
    What the model sees: context, instructions and chance
    strand
    how-it-works
    minutes
    30
    anchor cases
    • dpd-chatbot-2024
    objectives
    • id
      200.1-a
      text
      Tell apart what a chat assistant learned in training from what it is given in this run (its context: instructions, the conversation, pasted files and tool results), and explain that a long attachment may not all reach the model and that facts in the middle of a long context are more likely to be missed.
      misconception
      It remembers everything I have told it and reads every page I paste equally carefully.
      quiz item
      A Big Widget planner pastes a 90-page supplier contract and the answer misses a clause on page 45; use the supplied trace of the model's actual input (after the tool's parsing and retrieval) to pick whether the clause never reached the model (cut off at the file limit) or reached it and was missed, then pick the fix and check the clause in the contract itself, not in the model's quote of it.
    • id
      200.1-b
      text
      Explain why the same request can give different answers (when a tool samples each next token (small piece of text), the same request can lead to different answers; how much depends on settings that may be unknown to the user), and decide when to compare several answers and when to pin the task down instead.
      misconception
      A different answer the second time means the first one was a bug.
      quiz item
      Two runs of the same prompt give two different supplier summaries; choose what that does and does not tell you.
    • id
      200.1-c
      text
      Explain why instructions in a system prompt are not a security boundary, using a made-up Big Widget dealer bot, and say what the DPD case does and does not show (the BBC reports what the bot said and DPD's statement that an error followed a system update; which control failed was not reported).
      misconception
      If the bot's instructions forbid it, the bot cannot do it.
      quiz item
      For a made-up Big Widget dealer bot and a hostile chat, pick the control that sits outside the model and why a stronger instruction alone would not hold; then pick the one sentence about DPD that the BBC report supports (not a claim about which control failed).
    starting sources
    • Liu et al. 2023, Lost in the Middle (arXiv 2307.03172; TACL 2024)
    • Holtzman et al. 2019, The Curious Case of Neural Text Degeneration (arXiv 1904.09751; ICLR 2020)
    • NCSC, Prompt injection is not SQL injection (it may be worse), 2025
    • DPD chatbot, Jan 2024: BBC News, 19 Jan 2024, quoting DPD's statement (find DPD's own statement or the customer's post)
  • id
    200.2
    slug
    prompts-that-hold-up
    title
    Prompts that hold up
    strand
    building
    minutes
    35
    anchor cases
    • support-agents-2023
    objectives
    • id
      200.2-a
      text
      Write a reusable prompt template for a recurring work task that states the job, the audience, the facts to use, an example, the output format, and what to do when unsure.
      misconception
      Good prompting is a knack that cannot be written down and reused.
      quiz item
      Pick which of four Big Widget service-email prompts will give a checkable answer on the hundredth customer, not just the first.
    • id
      200.2-b
      text
      Break a multi-step job into steps and ask for working you can check on your own (numbers, quotes, citations), then check each step against the source or a calculator, knowing that the written working is not a reliable account of how the model reached its answer.
      misconception
      If it shows its working, that is how it got the answer, and the answer is right.
      quiz item
      A shown calculation of warranty cost has a wrong middle step but a plausible total; spot the step to recheck with a calculator, and pick why the shown working alone does not prove the total.
    • id
      200.2-c
      text
      Test a prompt on several realistic inputs, including awkward ones, using made-up or approved and redacted examples rather than live customer or company records, before relying on it.
      misconception
      If the prompt worked once, it works.
      quiz item
      Choose the smallest test that would show a dealer-reply prompt fails on complaints, not just on thank-you notes.
    starting sources
    • Wei et al. 2022, Chain-of-Thought Prompting (arXiv 2201.11903; NeurIPS 2022)
    • Brown et al. 2020, Language Models are Few-Shot Learners (arXiv 2005.14165), for examples in the prompt
    • Anthropic, Prompt engineering overview (Claude docs; as-of date needed)
    • Brynjolfsson, Li and Raymond 2023, Generative AI at Work (NBER w31161; QJE 2025)
  • id
    200.3
    slug
    grounding-it-in-your-documents
    title
    Grounding it in your documents
    strand
    building
    minutes
    35
    anchor cases
    • air-canada-2024
    objectives
    • id
      200.3-a
      text
      Explain what grounding changes (the model answers from documents you supply or a tool retrieves) and what it does not fix.
      misconception
      If I attach the document, the answer comes from the document.
      quiz item
      Pick which of three answers about a Big Widget returns policy is supported by the attached policy page, and which only sounds like it.
    • id
      200.3-b
      text
      Ask for answers that quote and point to the passage they used, and check each quote against the source.
      misconception
      A citation in the answer means the source says it.
      quiz item
      Given an answer, its cited passage and the real page, spot the claim the passage does not support.
    • id
      200.3-c
      text
      Spot when a grounded answer goes wrong because published documents contradict each other, are out of date or are missing, name who owns fixing them, and know when to escalate a wrong answer whose documents look right (tracing whether retrieval or generation failed is Level 300).
      misconception
      Wrong answers are always the model's fault, never the documents'.
      quiz item
      In Air Canada, the chatbot's answer contradicted the airline's own policy page and the tribunal held the airline responsible for both; choose who must reconcile what the company publishes. Then, for a made-up Big Widget answer, pick whether the fault is in the source documents or whether it should be escalated to the tool's owner.
    starting sources
    • Lewis et al. 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401; NeurIPS 2020)
    • Liu et al. 2023, Lost in the Middle (arXiv 2307.03172; TACL 2024)
    • Moffatt v. Air Canada, 2024 BCCRT 149 (the tribunal's own decision site)
    • NIST AI 600-1 (2024), Generative AI Profile (information integrity, confabulation)
  • id
    200.4
    slug
    testing-output-before-you-trust-it
    title
    Testing output before you trust it
    strand
    verifying
    minutes
    40
    anchor cases
    • mata-v-avianca-2023
    objectives
    • id
      200.4-a
      text
      Build a small test set for one task from the user's side (realistic examples with known good answers, including hard ones, made up or approved and redacted) and score the tool's outputs against a written rubric of what a right and usable answer is. Tracing which stage failed is Level 300.
      misconception
      You can judge an AI tool by trying it a few times.
      quiz item
      Pick the best 10-item test set for a tool that drafts quality-defect summaries.
    • id
      200.4-b
      text
      Explain the known biases of using one model to grade another's output (position, length, self-preference) and decide when a person must check instead.
      misconception
      Having a second AI check the answer is as good as a person checking it.
      quiz item
      An AI grader prefers the longer of two answers; choose what to change in the test.
    • id
      200.4-c
      text
      Decide how much checking a task needs from the cost of an error and who it lands on, from spot checks to checking every output.
      misconception
      Every AI output needs the same amount of checking.
      quiz item
      Rank four Big Widget tasks (meeting notes, a recall notice, a dealer price quote, a lunch menu) by how much checking they need, and say why.
    starting sources
    • Zheng et al. 2023, Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (arXiv 2306.05685; NeurIPS 2023 Datasets and Benchmarks)
    • Liang et al. 2022, Holistic Evaluation of Language Models (arXiv 2211.09110; TMLR 2023)
    • NIST AI 600-1 (2024), Generative AI Profile
    • Mata v. Avianca, No. 22-cv-1461 (PKC), Opinion and Order on Sanctions, S.D.N.Y., 22 Jun 2023 (docket entry 54)
  • id
    200.5
    slug
    protecting-data-in-prompts-files-and-connectors
    title
    Protecting data in prompts, files and connectors
    strand
    governing
    minutes
    40
    anchor cases
    • garante-openai-2024
    • echoleak-2025
    objectives
    • id
      200.5-a
      text
      Before sharing data with an AI tool, classify it with the company's data rules, then check that the tool is approved for that class and purpose and that its settings (training use, retention) and connector permissions fit; say "not enough information" when any of these is unknown.
      misconception
      If the company pays for the tool, any company data can go in it.
      quiz item
      Given Big Widget's (made-up) data classes, approved-tools list and the settings and connectors of three tools, pick where a customer complaint with a name and address may go, including the option "cannot tell from this".
    • id
      200.5-b
      text
      Explain how instructions hidden in an email, file or web page can make a connected assistant leak data (indirect prompt injection), and name two limits that reduce the risk.
      misconception
      Only what I type can give the assistant instructions.
      quiz item
      Using EchoLeak, choose which two limits would have reduced what one crafted email could expose.
    • id
      200.5-c
      text
      Describe what a data-protection regulator said it found in a chatbot's handling of personal data, dated to its announcement, and keep that separate from what happened to the decision later.
      misconception
      A regulator's headline fine is the final word.
      quiz item
      Given the Garante's December 2024 announcement and the note later added to its page on the Rome court ruling, pick the sentence for a staff briefing that dates both and does not claim the decision's current status.
    starting sources
    • Garante per la protezione dei dati personali, press release on provision no. 755 of 2 Nov 2024 (ChatGPT), with its note on Tribunale di Roma ruling no. 4153/2026
    • NVD, CVE-2025-32711 (M365 Copilot), published 11 Jun 2025
    • Greshake et al. 2023, Not what you've signed up for (arXiv 2302.12173; AISec 2023)
    • ICO, Guidance on AI and data protection
    • OWASP Top 10 for LLM Applications 2025
  • id
    200.6
    slug
    governed-agents
    title
    Governed agents: tools, permissions and approvals
    strand
    building
    minutes
    40
    folds in
    Learn 102 Governed agents
    anchor cases
    • aisi-agent-incident-2026
    objectives
    • id
      200.6-a
      text
      List the consequential actions an AI agent or automation can take (send, change, buy, publish, delete) and rate each by who it affects and whether it can be undone.
      misconception
      An agent's risk depends on how clever it is, not on what it is able to do.
      quiz item
      For a Big Widget purchasing agent, pick which of five actions is the hardest to undo.
    • id
      200.6-b
      text
      Read an agent's tools, connectors and permissions (in a made-up admin view) and match each to the actions it makes possible, finding any access the job does not need.
      misconception
      Approving the agent once approves everything it does.
      quiz item
      In a made-up admin view of a Big Widget purchasing agent, pick which permission lets it pay a supplier and which permission the job does not need.
    • id
      200.6-c
      text
      Explain least privilege for an agent's tools and connectors (only the access the job needs, scoped and logged) using a documented incident.
      misconception
      Give it broad access so it does not get stuck.
      quiz item
      Using the AISI incident, choose which change limited what the agent could reach next time.
    starting sources
    • UK AI Security Institute, Incident Report: unsanctioned agent behaviour during cyber testing, 4 Aug 2026
    • Anthropic, Building effective agents, Dec 2024
    • Model Context Protocol specification, Security Best Practices (2025-06-18)
    • OWASP Top 10 for LLM Applications 2025 (excessive agency)
    • Yao et al. 2022, ReAct (arXiv 2210.03629; ICLR 2023)
  • id
    200.7
    slug
    checking-claims-about-ai-tools
    title
    Checking claims about AI tools
    strand
    verifying
    minutes
    35
    anchor cases
    • ftc-donotpay-2024
    objectives
    • id
      200.7-a
      text
      Turn a vendor's or a headline's claim about an AI tool into testable questions (which task, measured how, against what, out of how many, by whom).
      misconception
      A benchmark score tells me how the tool will do on my work.
      quiz item
      For the claim "cuts quality-report time by 60%", pick the three questions from a list of six that would test it.
    • id
      200.7-b
      text
      Explain, using the FTC's DoNotPay action (the complaint's allegations, kept apart from the January 2025 order that settled them), why a claim that a tool can replace a professional needs evidence of testing against that professional.
      misconception
      If a product is on sale, someone has checked that it works as claimed.
      quiz item
      Pick which missing step the FTC's complaint describes in DoNotPay's "robot lawyer" claim.
    • id
      200.7-c
      text
      Plan a small pilot on your own work, with a baseline and a measure chosen in advance, to check a claim before rolling a tool out.
      misconception
      If the team likes the tool, it is working.
      quiz item
      Choose the pilot design that could show a drafting tool did not save time.
    starting sources
    • FTC, FTC Announces Crackdown on Deceptive AI Claims and Schemes (Operation AI Comply), 25 Sep 2024 (DoNotPay section; archived copy)
    • FTC, In the Matter of DoNotPay, Inc., Decision and Order, Docket No. C-4812, issued 14 Jan 2025
    • Liang et al. 2022, Holistic Evaluation of Language Models (arXiv 2211.09110; TMLR 2023)
    • NIST AI RMF 1.0 (2023), MEASURE function
    • Brynjolfsson, Li and Raymond 2023, Generative AI at Work (NBER w31161; QJE 2025), as a measured study with a baseline
  • id
    200.8
    slug
    the-policy-gate
    title
    The policy gate: retrofit a rule
    strand
    governing
    minutes
    45
    reuses
    Practice 200 Retrofit worksheet (DPD, VH-049; Knight Capital, VH-056)
    anchor cases
    • dpd-chatbot-2024
    • knight-capital-2012
    objectives
    • id
      200.8-a
      text
      Explain, starting from what the BBC reported about DPD's chatbot and then using a made-up Big Widget bot (not a finding about DPD's systems), why what a bot may say or promise for the company must be a rule outside the model, tested against angry and tricky users before launch.
      misconception
      A better-worded prompt fixes a bot that says the wrong thing.
      quiz item
      Pick which of four controls is a rule outside the model, and which are just more instructions.
    • id
      200.8-b
      text
      Explain, using the SEC's Knight Capital order, why a change to a deployment or a rule is itself an action to gate (a second reviewer, a written procedure, a way to stop and roll back).
      misconception
      Only the AI needs a gate, not the change that ships it.
      quiz item
      From the SEC's findings, choose the missing control that would have caught the eighth server.
    • id
      200.8-c
      text
      Write and test a policy for one tool's three consequential actions (allow, hold for a named approver, deny, and deny anything unlisted), and say how you will stay at yes (receipts, review of rule changes) and recover to yes (stop, roll back, change the rule). This tested policy is the artifact Level 300 enforces.
      misconception
      Allow-by-default is fine until something goes wrong.
      quiz item
      Spot the gap in a draft policy that allows unlisted actions and has no named approver, then pick the test input that would show the gap.
    starting sources
    • Practice Level 200 Retrofit worksheet: https://practice.aigovops-foundation.com/levels/200-retrofit/WORKSHEET.html
    • SEC, In the Matter of Knight Capital Americas LLC, Release No. 34-70694, 16 Oct 2013
    • DPD chatbot, Jan 2024: BBC News, 19 Jan 2024, quoting DPD's statement
    • NIST AI RMF 1.0 (2023), GOVERN and MANAGE
  • id
    200.R
    slug
    gen-ai-in-your-role
    title
    Gen AI in your role (role tracks)
    strand
    building
    minutes
    40
    tracks
    • home-and-everyday-life
    • manufacturing
    • quality
    • engineering-and-design
    • supply-chain-and-procurement
    • forecasting-and-planning
    • sales-and-dealers
    • customer-service
    • marketing
    • brand-and-communications
    • legal
    • it
    • cybersecurity
    • safety
    • hr
    • finance
    • leaders
    objectives
    • id
      200.R-a
      text
      For one recurring task in your role, write its prompt template, its grounding source, its check and its data rule.
      misconception
      Using gen AI well looks the same in every job.
      quiz item
      Match four Big Widget roles to the check their task needs most (a citation check, a numbers check, a tone check, a legal review).
    • id
      200.R-b
      text
      Name which AI tools your team uses that act (send, change, buy, publish) rather than only draft, and who approves those actions.
      misconception
      Our AI tools only make suggestions.
      quiz item
      From a list of six team tools, pick the two that take actions and need an approver.
    starting sources
    • Reuses the case bank (levels/200/cases.yml): each track row names the case it draws on
  • id
    200.P
    slug
    retrofit-a-rule-on-a-tool-you-use
    title
    Project: retrofit a rule on a tool you use
    strand
    project
    minutes
    60
    reuses
    Practice 200 Retrofit worksheet
    objectives
    • id
      200.P-a
      text
      Bring a tested policy for one AI tool or agent you use, or for the made-up Big Widget tool provided for learners with no approved tool (three consequential actions rated, one held for a named approver, one denied, anything unlisted denied); a test set of at least ten inputs, made up or approved and redacted, including tricky ones, with results that show whether each rule decided as intended; and one share-safe receipt (no customer or restricted data) from a gated or checked use. A second person reviews it, and a host marks it on the Foundation's Thursday call.
      misconception
      A policy is finished when it is written, not when it has been tested.
      quiz item
      Five auto-graded scenarios on a sample policy and receipts (is an unlisted action denied; does each hold name an approver; which test input exposes a gap; is this receipt safe to share; what is the recover-to-yes step), plus the host's marking sheet, which judges whether the tests exposed failures and the rules decided as intended, not only whether each part is present.
    starting sources
    • Practice Level 200 Retrofit worksheet: https://practice.aigovops-foundation.com/levels/200-retrofit/WORKSHEET.html