Level 500 objectives

Objectives map signed: Ken Johnston, 2026-11-18 (by proxy: Ken gave his proxy for all levels to Bob Rapp, who approved and recorded it at 20:35 PST). Level 500 lessons are published.

level
500
name
Governance, safety and research
for
Advanced practitioners and researchers who have finished Level 400. Reads primary research; comfortable with statistics, code and standards text.
prerequisite
Level 400 (Engineering AI systems), finished
reading level
US grade 8 (plain gear)
setting
Big Widget Company, a made-up 100,000-person maker of complex, expensive, car-like widgets (used where a module needs an organisation to govern or attack)
signed by
Ken Johnston, 2026-11-18 (by proxy: Ken gave his proxy for all levels to Bob Rapp, who approved and recorded it at 20:35 PST)
decision record
decisions-2026-11-18.md
timing and scope
Bob approved the drafted timings and bounded tabletop, critique and capstone scope on 18 November 2026 at 14:54 PST; keep all minute values unchanged. Ken's map sign-off is still pending.
fixture policy
The 500.6/500.7 exercises and 500.9 tabletop use only synthetic fixtures owned by this course repo, to be authored with the modules after map sign-off, not now. No private-repository excerpts, copied private vectors or learner dependency on private access. Model original synthetic records, rule cards and transcripts on the pinned public facts in the packs; private One Receipt references are for the map author and checker only. Independently check the missing-proof, signature-integrity and stop/coverage assumptions and keys before teaching; do not claim One Receipt conformance or deployment evidence.
prep rails
  • id
    500.rail-ai-assurance
    rail
    An AI governance or management-system audit certification practice rail
    vendor
    not cleared
    exam
    not cleared
    draws on
    • 500.6-a
    • 500.7-b
    • 500.8-b
    • 500.9-d
    • 500.R-a
    • 500.R-b
    practice
    Produce a role-specific assurance handoff, inspect a missing-evidence packet and route an unresolved finding to an independent owner; map each practice record to the listed objectives.
    limits
    Bob's decision is to keep every vendor and exam name unnamed until he clears each one. No current exam alignment is claimed. A passed exam concerns a person, not system certification.
modules
  • id
    500.1
    slug
    reading-primary-research
    title
    Reading and critiquing primary research
    strand
    verifying
    minutes
    50
    gear minutes
    plain
    50
    standard
    70
    technical
    90
    anchor cases
      objectives
      • id
        500.1-a
        text
        Critique an AI paper's headline claim against its evidence, naming the baseline, the sample, the metric and what the result would look like if the claim were false.
        misconception
        A result in a paper is a fact; the abstract says what the paper showed.
        quiz item
        Given an abstract and its results table, pick the one claim the table does not support.
      • id
        500.1-b
        text
        Classify a source as peer-reviewed paper, preprint, vendor report or system card, and say what each status does and does not establish.
        misconception
        Peer review, or an arXiv number, means the claim is true.
        quiz item
        Match four sources to their status and choose the right statement of what each status guarantees.
      • id
        500.1-c
        text
        Spot leakage, test-set contamination and construct-validity problems in a benchmark result before relying on it.
        misconception
        A higher benchmark score means the model is better at the real job.
        quiz item
        From a short method description, identify which of three flaws (leakage, contamination, construct validity) undermines the score.
      starting sources
      • Lipton, Zachary C. et al. (2019). Troubling Trends in Machine Learning Scholarship; ACM Queue 17(1), February 2019 (commentary; not a peer-reviewed research paper) — https://api.crossref.org/works/10.1145/3317287.3328534 (500.1-s1).
      • Kapoor, Sayash et al. (2023). Leakage and the Reproducibility Crisis in ML-based Science; Patterns (2023) — https://api.crossref.org/works/10.1016/j.patter.2023.100804 (500.1-s2).
      • Raji, Inioluwa Deborah et al. (2021). AI and the Everything in the Whole Wide World Benchmark; NeurIPS 2021 Datasets and Benchmarks Track (Round 2) — https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/hash/084b6fbb10729ed4da8c3d3f5a3ae7c9-Abstract-round2.html (500.1-s3).
      • Biderman, Stella et al. (2024). Lessons from the Trenches on Reproducible Evaluation of Language Models — https://arxiv.org/abs/2405.14782 (500.1-s4).
      • Sainz, Oscar et al. (2023). NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark — https://arxiv.org/abs/2310.18018 (500.1-s5).
      • Joelle Pineau et al. (2021). Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) — https://jmlr.org/papers/v22/20-303.html (500.1-s6).
      • Lipton, Zachary C. et al. (2018). Troubling Trends in Machine Learning Scholarship; preprint (not peer-reviewed) — https://arxiv.org/abs/1807.03341 (500.1-s7).
    • id
      500.2
      slug
      what-research-says-about-how-models-work
      title
      What research can and cannot yet say about how models work
      strand
      how-it-works
      minutes
      50
      gear minutes
      plain
      50
      standard
      70
      technical
      90
      anchor cases
        builds on
        400.1 scaling baseline and 400.6 evaluation uncertainty; supplied excerpts and pre-scored plots, not model training or fresh benchmark runs.
        scope
        Plain gear uses one scaling comparison, two pre-scored emergence plots, one chain-of-thought result and one toy-mechanism diagram; deeper replication is optional out-of-class work.
        objectives
        • id
          500.2-a
          text
          Building on 400.1, compare two supplied scaling-law summaries and state one prediction about loss and one unsupported inference about a specific ability.
          misconception
          Scaling laws tell you which new abilities a bigger model will have.
          quiz item
          Choose which of four forecasts a scaling-law fit can and cannot support.
        • id
          500.2-b
          text
          Critique an emergence claim using two supplied, pre-scored plots of the same results under different metrics; name the metric sensitivity and what remains unresolved about the ability.
          misconception
          A sudden jump on a chart proves a new ability appeared at that scale.
          quiz item
          Given two pre-scored plots, select the bounded critique; include an unresolved conclusion rather than forcing a model-versus-metric binary.
        • id
          500.2-c
          text
          Identify the supported conclusion and one limit of a supplied chain-of-thought faithfulness experiment.
          misconception
          The model's written reasoning is necessarily the real reason for its answer.
          quiz item
          Pick the conclusion a chain-of-thought faithfulness experiment supports.
        • id
          500.2-d
          text
          From a supplied toy-superposition diagram and study summary, distinguish evidence about the toy mechanism from an unsupported explanation of a deployed model's decision.
          misconception
          A mechanism demonstrated in a toy network explains any deployed model's behaviour.
          quiz item
          Sort a toy-network finding and a deployed-model causal claim into supported, not established or unresolved by this experiment.
        starting sources
        • Kaplan, Jared et al. (2020). Scaling Laws for Neural Language Models — https://arxiv.org/abs/2001.08361 (500.2-s1).
        • Hoffmann, Jordan et al. (2022). Training Compute-Optimal Large Language Models — https://arxiv.org/abs/2203.15556 (500.2-s2).
        • Schaeffer, Rylan et al. (2023). Are Emergent Abilities of Large Language Models a Mirage?; NeurIPS 2023 — https://proceedings.neurips.cc/paper_files/paper/2023/hash/adc98a266f45005c403b8311ca7e8bd7-Abstract-Conference.html (500.2-s3).
        • Turpin, Miles et al. (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting; NeurIPS 2023 — https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract-Conference.html (500.2-s4).
        • Nelson Elhage et al. (2022). Toy Models of Superposition — https://transformer-circuits.pub/2022/toy_model/index.html (500.2-s5).
      • id
        500.3
        slug
        alignment-and-its-limits
        title
        Alignment and its limits
        strand
        how-it-works
        minutes
        55
        gear minutes
        plain
        55
        standard
        75
        technical
        95
        anchor cases
          objectives
          • id
            500.3-a
            text
            Explain how optimising a proxy (a reward model, a test, a score) can produce specification gaming or reward hacking, with a documented example.
            misconception
            A trained model does what its trainers intended.
            quiz item
            Given a reward and a behaviour, say whether the behaviour games the specification and why.
          • id
            500.3-b
            text
            Name the open problems in learning from human feedback that limit how far it can make a model safe.
            misconception
            Fine-tuning with human feedback solves alignment.
            quiz item
            Choose which listed limitation applies to a described RLHF failure.
          • id
            500.3-c
            text
            Critique a behavioural-safety claim by stating what the test could not observe, using published studies of behaviours that persisted through safety training, with each study's own stated limits.
            misconception
            If safety training removed the behaviour in testing, the behaviour is gone.
            quiz item
            From a study summary, pick the conclusion the authors themselves say it does not support.
          starting sources
          • Amodei, Dario et al. (2016). Concrete Problems in AI Safety — https://arxiv.org/abs/1606.06565 (500.3-s1).
          • Casper, Stephen et al. (2023). Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback — https://arxiv.org/abs/2307.15217 (500.3-s2).
          • Hubinger, Evan et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training — https://arxiv.org/abs/2401.05566 (500.3-s3).
          • Greenblatt, Ryan et al. (2024). Alignment faking in large language models — https://arxiv.org/abs/2412.14093 (500.3-s4).
          • Victoria Krakovna et al. (2020). Specification gaming: the flip side of AI ingenuity — https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/ (500.3-s5).
        • id
          500.4
          slug
          evaluating-frontier-risk
          title
          Evaluating frontier risk
          strand
          verifying
          minutes
          50
          gear minutes
          plain
          50
          standard
          70
          technical
          90
          anchor cases
          • aisi-agent-incident-2026
          objectives
          • id
            500.4-a
            text
            Explain what a dangerous-capability evaluation measures, and why a failed elicitation is not proof the capability is absent.
            misconception
            If the model failed the evaluation, it cannot do the thing.
            quiz item
            Given an evaluation report, choose the strongest conclusion it licenses.
          • id
            500.4-b
            text
            Read a published frontier safety framework and state which evidence would show a commitment was kept or broken.
            misconception
            A published safety policy is evidence that a model is safe.
            quiz item
            For one commitment in an excerpt, pick the evidence an outsider could check.
          • id
            500.4-c
            text
            Critique a capability-trend or incident claim by its method, scope and stated limits before repeating it.
            misconception
            A trend line or an incident report tells you how the model behaves everywhere.
            quiz item
            Choose the scope statement that matches what an incident report actually says.
          starting sources
          • Shevlane, Toby et al. (2023). Model evaluation for extreme risks — https://arxiv.org/abs/2305.15324 (500.4-s1).
          • Bengio, Yoshua et al. (2025). International AI Safety Report — https://arxiv.org/abs/2501.17805 (500.4-s2).
          • Anthropic (2026). Anthropic’s Responsible Scaling Policy — https://www.anthropic.com/responsible-scaling-policy (500.4-s3).
          • UK AI Security Institute (2026). Incident Report: unsanctioned agent behaviour during cyber testing — https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing (500.4-s4).
          • Kwa, Thomas et al. (2025). Measuring AI Ability to Complete Long Software Tasks; NeurIPS 2025 — https://neurips.cc/virtual/2025/poster/119302 (500.4-s5).
        • id
          500.5
          slug
          building-safeguards-that-hold
          title
          Building safeguards that hold under attack
          strand
          building
          minutes
          55
          gear minutes
          plain
          55
          standard
          75
          technical
          95
          anchor cases
            objectives
            • id
              500.5-a
              text
              Design layered safeguards for a Big Widget Company agent with tool access, and name the attack each layer cannot stop.
              misconception
              A strong system prompt or an input filter stops prompt injection.
              quiz item
              Given a layered design and an indirect-injection attack, identify the first layer that actually blocks it, or that none does.
            • id
              500.5-b
              text
              Explain why adversarial robustness is unsolved, using published transferable or automated attacks on aligned models.
              misconception
              Patching the jailbreak that worked fixes the class of attack.
              quiz item
              Choose the statement about attack transfer that a cited study supports.
            starting sources
            • Greshake, Kai et al. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — https://arxiv.org/abs/2302.12173 (500.5-s1).
            • Zou, Andy et al. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models — https://arxiv.org/abs/2307.15043 (500.5-s2).
            • Debenedetti, Edoardo et al. (2025). Defeating Prompt Injections by Design — https://arxiv.org/abs/2503.18813 (500.5-s3).
            • Apostol Vassilev et al. (2025). Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2025) — https://csrc.nist.gov/pubs/ai/100/2/e2025/final (500.5-s4).
            • OWASP GenAI Security Project (2025). OWASP Top 10 for LLM Applications 2025 — https://genai.owasp.org/llm-top-10/ (500.5-s5).
          • id
            500.6
            slug
            one-receipt
            title
            One Receipt: evidence a stranger can verify
            strand
            governing
            minutes
            50
            gear minutes
            plain
            50
            standard
            70
            technical
            90
            reuses
            The 400.8/400.9/400.P Beacon bundle contract with VERIFY.md, represented by a course-owned synthetic bundle to be authored with the module. Use pinned public RFC 9943, RFC 9942, RFC 8785 and W3C VC 2.0 facts for original synthetic receipt/rule cards, not private One Receipt excerpts or vectors. The pinned One Receipt v0.2-draft reference remains author/checker context only.
            anchor cases
            • air-canada-2024
            • knight-capital-2012
            objectives
            • id
              500.6-a
              text
              Using the course-owned synthetic Beacon-style bundle and original synthetic receipt cards, distinguish signature integrity and origin under stated key/trust assumptions from an issuer's attestation and from actual control effectiveness.
              misconception
              A signed, verified receipt proves the AI's output was true, fair or lawful.
              quiz item
              Given a synthetic signed record and explicit key/trust assumptions, pick the supported integrity/origin statement without inferring truth or control effectiveness.
            • id
              500.6-b
              text
              Compare a course-owned synthetic missing-registry-evidence case with its original fictional assurance rule card; record why the course verifier withholds the higher fictional assurance when required proof is absent, without inferring truth or control effectiveness.
              misconception
              The issuer's claimed assurance level is the level you get.
              quiz item
              Given the synthetic case and its fictional course rule card, choose the capped result and missing proof; this key is not a One Receipt conformance result, and registration does not certify the claim.
            • id
              500.6-c
              text
              Design the content-free receipt evidence for one consequential Big Widget Company AI action, using commitments instead of exported prompts or outputs.
              misconception
              Exporting prompts and outputs is necessary, and a signature proves the declared control was effective.
              quiz item
              Choose the content-free evidence fields and identify which control-effectiveness claim still needs independent observation.
            starting sources
            • AiGovOps Foundation Technical Incubator (2026). One Receipt — Core Specification (v0.2-draft) — https://github.com/aigovops-foundation/One-ai-Receipt-aigovops-foundation/blob/a8385049205bb85690c5db6a1e356930a7d719f1/spec/ONE-RECEIPT-SPEC.md (500.6-s1).
            • H. Birkholz et al. (2026). An Architecture for Trustworthy and Transparent Digital Supply Chains (RFC 9943) — https://www.rfc-editor.org/rfc/rfc9943.html (500.6-s2).
            • O. Steele et al. (2026). CBOR Object Signing and Encryption (COSE) Receipts (RFC 9942) — https://www.rfc-editor.org/rfc/rfc9942.html (500.6-s3).
            • A. Rundgren et al. (2020). JSON Canonicalization Scheme (JCS) (RFC 8785) — https://www.rfc-editor.org/rfc/rfc8785.html (500.6-s4).
            • Manu Sporny et al. (2025). Verifiable Credentials Data Model v2.0 — https://www.w3.org/TR/2025/REC-vc-data-model-2.0-20250515/ (500.6-s5).
            • U.S. Securities and Exchange Commission (2013). In the Matter of Knight Capital Americas LLC, Release No. 34-70694 — https://web.archive.org/web/20240714045144id_/https://www.sec.gov/files/litigation/admin/2013/34-70694.pdf (500.6-s6).
          • id
            500.7
            slug
            overt-and-the-sail-crosswalk
            title
            OVERT and the SAIL crosswalk
            strand
            governing
            minutes
            50
            gear minutes
            plain
            50
            standard
            70
            technical
            90
            reuses
            Course-owned synthetic Beacon-style bundle with VERIFY.md and original synthetic attestation/policy cards, to be authored with the module using public OVERT 1.1 (11 June 2026), SAIL risk IDs and the pinned public publisher licence. The private pinned crosswalk and vector 13 remain author/checker sources, not learner inputs or copied fixtures. Stewardship statements are as observed 18 November 2026, not an assertion of effective transfer.
            anchor cases
              objectives
              • id
                500.7-a
                text
                Compare public OVERT 1.1 principles and course-authored synthetic rule cards against the synthetic Beacon-style bundle, separating signature integrity/origin, attested runtime-control claims and independently tested control effectiveness.
                misconception
                OVERT conformance means the AI system is safe.
                quiz item
                Choose the claim an OVERT-conformant runtime record can support.
              • id
                500.7-b
                text
                Trace one public SAIL risk id through a course-owned synthetic attested control claim; record the missing evidence needed to test actual effectiveness and a risk the receipt cannot cover by design.
                misconception
                A crosswalk row means the control works.
                quiz item
                Given a course-owned synthetic signed record with its signed bytes and evidence references unchanged, but only the referenced external evidence unavailable, pick the conclusion that signature verification can still pass under the supplied key/trust assumptions while control execution and effectiveness remain unestablished; do not treat this as a modified signed record or use private vector 13.
              • id
                500.7-c
                text
                Distinguish a course-authored synthetic cite-by-id-only editorial policy from the publisher repository's versioned CC BY-NC-SA 4.0 terms, marking scope over other assets or separate permissions as not checked; the private crosswalk remains author/checker context, not a learner excerpt.
                misconception
                The crosswalk's conservative editorial rule is the complete publisher licence.
                quiz item
                Classify statements as the course's synthetic editorial policy, a condition in the supplied public versioned licence, or permission not established; do not infer rights over other assets or require private crosswalk access.
              starting sources
              • Ken Johnston et al. (2026). What is OVERT? — https://www.aigovops-foundation.com/overt.html (500.7-s1).
              • Glacis Technologies, Inc. (2026). OVERT — Observable Verification Evidence for Runtime Trust (v1.1) — https://overt.is/ (500.7-s2).
              • AiGovOps Foundation (2026). Crosswalk — SAIL v2 risks ↔ One Receipt evidence — https://github.com/aigovops-foundation/One-ai-Receipt-aigovops-foundation/blob/a8385049205bb85690c5db6a1e356930a7d719f1/spec/crosswalks/sail-v2.md (500.7-s3).
              • AiGovOps Foundation (2026). Delta matrix — what One Receipt takes from, gives to, and leaves alone — https://github.com/aigovops-foundation/One-ai-Receipt-aigovops-foundation/blob/a8385049205bb85690c5db6a1e356930a7d719f1/spec/delta-matrix.md (500.7-s4).
              • Pillar Security (2026). SAIL (Secure AI Lifecycle) Framework V2 — https://www.pillar.security/sail (500.7-s5).
              • Pillar Security (pillar-labs) (2026). SAIL skill licence — Attribution-NonCommercial-ShareAlike 4.0 International — https://github.com/pillar-labs/sail-skill/blob/67e1691ac64586b0abea96c74f511ea5628f62ed/LICENSE (500.7-s6).
              • OWASP GenAI Security Project (2025). OWASP Top 10 for Agentic Applications for 2026 — https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ (500.7-s7).
            • id
              500.8
              slug
              designing-a-governance-regime
              title
              Designing a governance regime
              strand
              governing
              minutes
              60
              gear minutes
              plain
              60
              standard
              80
              technical
              100
              anchor cases
              • robodebt-2023
              • toeslagenaffaire-2020
              scenario
              Fictional exercise dated 18 November 2026. Big Widget Company's EU subsidiary uses a purchased hiring assistant under its authority to rank EU job applicants for HR review; procurement calls it a deployer. Validate that label and any additional provider capacity rather than accepting procurement's conclusion. The packet does not state who developed or commissioned the system, whose name or trademark it entered service under, or whether configuration changed its intended purpose or substantially modified it; identify the role facts still needed. The HR governance lead owns gates before ranking use, rejection and challenge resolution. Applicability is a question to document, not a legal conclusion supplied by the scenario.
              builds on
              Reuse 300.7-a/b on origin/devin/1795003421-level300-map for as-of-date applicability and provider/deployer-or-both analysis under Articles 3 and 25; identify missing facts instead of inferring a sole role from purchase or use.
              legal basis
              Compare the original Regulation (EU) 2024/1689 with Regulation (EU) 2026/1744 of 8 July 2026, Article 1(40), amending Article 113, using the version-specific Internet Archive copies in 500.8-s7. For the scenario date, record each relevant provision's application date and scope rather than treating the original 2024 timeline or Commission guidance as legal authority. The amendment's Article 1(40)(b) defers Chapter III Sections 1–3, except Article 6(5), to 2 December 2027 for Article 6(2)/Annex III systems and 2 August 2028 for Article 6(1)/Annex I systems; do not infer that all AI Act duties are deferred or that a future duty is already applicable. Check classification and any transitional facts separately.
              objectives
              • id
                500.8-a
                text
                Design a governance regime for the Big Widget Company that names who decides, the gate before each consequential action, the evidence kept and the route to challenge and remedy.
                misconception
                Governance means adopting a framework document.
                quiz item
                Given a regime with a missing element, identify which (decider, gate, evidence, remedy) is missing.
              • id
                500.8-b
                text
                For the dated hiring-assistant scenario, reuse 300.7-a/b to validate the stated provider/deployer role or both against the facts, name missing role facts, and produce a clause → gate → evidence → owner table citing version-specific original and amending legal text with application dates and scope uncertainties; distinguish NIST guidance and ISO catalogue scope, leaving ISO/IEC 42001 clauses not checked.
                misconception
                Buying a system makes the buyer only a deployer; the original 2024 timeline or voluntary guidance establishes the duties applicable on a later date.
                quiz item
                Select the table row that tests procurement's deployer label under Articles 3 and 25, leaves additional provider capacity unresolved pending missing branding/development/modification facts, and cites the applicable original/amending clause and application date rather than guidance; reject an ISO catalogue entry presented as a verified clause or certification.
              • id
                500.8-c
                text
                Explain, using a documented public-sector failure, why contestability and an end-to-end audit trail are design requirements, not after-the-fact fixes.
                misconception
                An audit after deployment is enough to catch harm.
                quiz item
                From a supported report recommendation and a proposed challenge route, identify which documented barrier the route addresses; no claim that it would have changed the historical outcome.
              starting sources
              • Elham Tabassi (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0) — https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10 (500.8-s1).
              • Raji, Inioluwa Deborah et al. (2020). Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing — https://arxiv.org/abs/2001.00973 (500.8-s2).
              • European Commission, Directorate-General for Communications Networks, Content and Technology (2026). AI Act — https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai (500.8-s3).
              • Parlementaire ondervragingscommissie Kinderopvangtoeslag, Tweede Kamer der Staten-Generaal (2020). Ongekend onrecht — https://www.tweedekamer.nl/sites/default/files/atoms/files/20201217_eindverslag_parlementaire_ondervragingscommissie_kinderopvangtoeslag.pdf (500.8-s4).
              • National Institute of Standards and Technology (2026). NIST AI RMF Playbook — https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook (500.8-s5).
              • Royal Commission into the Robodebt Scheme (2023). Report of the Royal Commission into the Robodebt Scheme — https://web.archive.org/web/20261027062045/https://robodebt.royalcommission.gov.au/publications/report (500.8-s6).
              • European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act) — https://web.archive.org/web/20261117042205/https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng (500.8-s7).
              • European Parliament and Council of the European Union (8 July 2026). Regulation (EU) 2026/1744, Article 1(40) amending Article 113 and Article 4 entry into force; Internet Archive copy — https://web.archive.org/web/20261116050222/https://eur-lex.europa.eu/eli/reg/2026/1744/oj/eng (500.8-s7 amendment).
              • International Organization for Standardization and International Electrotechnical Commission (2023). ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system — https://web.archive.org/web/20261006083019/https://www.iso.org/standard/81230.html (500.8-s8).
            • id
              500.9
              slug
              running-a-red-team-exercise
              title
              Running a red-team exercise
              strand
              governing
              minutes
              60
              gear minutes
              plain
              60
              standard
              80
              technical
              100
              reuses
              This course's own red-team process (agents/red-team.md, TRUTH-STANDARD.md rule 3)
              anchor cases
              • aisi-agent-incident-2026
              objectives
              • id
                500.9-a
                text
                Plan a red-team exercise on a Big Widget Company AI system with scope, threat model, rules of engagement, stop conditions and a named human lead who rules on findings.
                misconception
                Red teaming means trying jailbreaks until one works.
                quiz item
                Choose which missing plan element (scope, rules of engagement, stop condition, ruling lead) makes a described exercise unsafe.
              • id
                500.9-b
                text
                Write a finding that another person can reproduce, rate and rule on.
                misconception
                A screenshot of a bad output is a finding.
                quiz item
                Pick the finding write-up that a red-team lead could rule on.
              • id
                500.9-c
                text
                Explain the limits of red teaming as evidence, including coverage, and why finding nothing does not show a system is safe.
                misconception
                A red team that found nothing proves the system is safe.
                quiz item
                Choose the conclusion a clean red-team report licenses.
              • id
                500.9-d
                text
                Run a short authorised tabletop on course-owned synthetic Big Widget Company transcripts authored in this repo; record tested and untested coverage, apply stop conditions, and obtain and record a finding ruling by a named person other than the tester.
                misconception
                Writing a red-team plan or grading your own findings counts as running an independently ruled exercise.
                quiz item
                Given a tabletop log, choose the valid stop action and coverage statement, and identify the ruling that must be returned because the tester ruled on their own finding.
              assessment
              Mandatory for every learner regardless of capstone route. Grade the run record on authorisation, stop-condition handling, tested/untested coverage and an independent ruling; no live targets, credentials or external actions. Bob approved this bounded scope and the drafted timings on 18 November 2026 at 14:54 PST. Use only course-owned synthetic transcript cards to be authored with the module, never private-repo excerpts; the human ruling may be completed after class and is required before completion.
              starting sources
              • Ganguli, Deep et al. (2022). Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned — https://arxiv.org/abs/2209.07858 (500.9-s1).
              • Perez, Ethan et al. (2022). Red Teaming Language Models with Language Models — https://arxiv.org/abs/2202.03286 (500.9-s2).
              • Feffer, Michael et al. (2024). Red-Teaming for Generative AI: Silver Bullet or Security Theater? — https://arxiv.org/abs/2401.15897 (500.9-s3).
              • Bullwinkel, Blake et al. (2025). Lessons From Red Teaming 100 Generative AI Products — https://arxiv.org/abs/2501.07238 (500.9-s4).
              • UK AI Security Institute (2026). Incident Report: unsanctioned agent behaviour during cyber testing — https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing (500.9-s5).
            • id
              500.R
              slug
              research-and-governance-roles
              title
              Research and governance roles (role tracks)
              strand
              building
              minutes
              45
              gear minutes
              plain
              45
              standard
              60
              technical
              75
              tracks
              • researcher
              • auditor-or-assessor
              • regulator-or-policy
              • red-team-lead
              • standards-editor
              • governance-lead
              • reviewer
              objectives
              • id
                500.R-a
                text
                Produce a role-specific assurance handoff from a supplied evidence packet, with a bounded confidence statement, unresolved finding, check instructions and a named independent next owner.
                misconception
                Expert work is complete when an expert names a question or expresses confidence without a checkable handoff.
                quiz item
                For the assigned role, choose the handoff whose evidence changes confidence while keeping open questions unresolved, and whose unresolved finding has an independent owner.
              • id
                500.R-b
                text
                Map your handoff practice to the unnamed assurance prep rail and its course objectives, and distinguish evidence about a person's preparation from evidence about a system's controls.
                misconception
                Finishing a prep rail grants a certification or certifies a system.
                quiz item
                Match a handoff check to a mapped rail objective and separate a person's exam result from system evidence; no vendor-proctored exam or endorsement is supplied by this course.
              role practice
              Researchers hand off a replication discrepancy; assessors a missing-evidence register; policy staff a scope-exception memo; red-team leads a ruling and retest handoff; standards editors a versioned ambiguity/test-vector note; governance leads an escalation record; reviewers an evidence/revision disposition. Use existing packets rather than repeating a whole literature critique or regime design.
              starting sources
              • National Institute of Standards and Technology (2026). NIST AI RMF Playbook — https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook (500.R-s1).
              • Elham Tabassi (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0) — https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10 (500.R-s2).
              • Raji, Inioluwa Deborah et al. (2020). Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing — https://arxiv.org/abs/2001.00973 (500.R-s3).
              • AiGovOps Foundation (2026). Practitioner program — OR-100 to OR-600 — https://github.com/aigovops-foundation/One-ai-Receipt-aigovops-foundation/blob/main/docs/PRACTITIONER-PROGRAM.md (500.R-s4).
              • Joelle Pineau et al. (2021). Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) — https://jmlr.org/papers/v22/20-303.html (500.R-s5).
            • id
              500.P
              slug
              capstone
              title
              Capstone: publish work that survives review
              strand
              project
              minutes
              120
              gear minutes
              plain
              120
              standard
              120
              technical
              120
              reuses
              Practice 400 Prove it worksheet; the Foundation's free PhD-level program capstone (reproduce one paper and publish what held)
              in gear
              Prepare one artifact from supplied data or a carried-forward draft and complete its pre-submission evidence/privacy checklist within the existing 120 minutes; full replication, external review and release are not included in that time.
              common rubric
              Every route states a bounded claim, a falsifier or failure condition, provenance and stranger-check instructions, and limitations including missing evidence and uncertainty. Pin the exact artifact and evidence version with immutable revision identifiers or a file-hash manifest; every gate record identifies that same version. A gate for an older version does not automatically clear a changed version.
              route checks
              paper-reproduction
              For the exact artifact/evidence version, pin paper/data/code/environment, reproduce one bounded result against a baseline and report mismatches and uncertainty. If full reproduction is inaccessible, define a narrower claim supported by an independently runnable substitute, such as a check on a frozen accessible subset or released artifact, with pinned inputs, environment, baseline, expected outputs and stranger-run instructions. Someone other than the author must actually rerun that substitute and record results for this version; do not generalise its result to the full paper. If no substitute exists, document the limitation and keep the artifact private and draft; a limitation alone never satisfies the independent-rerun gate.
              governance-regime
              For the exact artifact/evidence version, supply the dated role/use, legal-scope assumptions, clause → gate → evidence → owner table, a missing-evidence test and a challenge/remedy walkthrough; no certification claim from catalogue scope.
              red-team-report
              For the exact artifact/evidence version, supply authorisation, reproducible fictional transcript tests, stop/coverage log, independently recorded ruling, mitigation retest and residual limitations; the tester cannot rule on their own finding.
              out of class steps
              • Pin the exact artifact and evidence version, then have someone other than the author replicate or rerun the route's check instructions for that version, recording results and disagreements. The paper route may use its defined narrower runnable substitute, but an access limitation without an independent rerun leaves this gate unmet and the artifact private and draft.
              • Independent red team by someone other than the author for that exact artifact/evidence version, with a version-bound ruling recorded by a named person other than the tester.
              • Close and retest every accepted red-team finding, with independent source re-check of affected claims and recorded closure, all identifying the exact artifact/evidence version before formal review; document residual limitations without leaving accepted findings open. Material changes during closure also reopen affected independent reruns and red-team rulings before formal review.
              • Formal two-human review of that exact artifact/evidence version, one subject expert and one audience reader, neither replaced by automated grading. After any material change, including a changed claim, baseline, data, code or evidence, pin the changed version and reopen affected independent reruns and red-team rulings, close and retest accepted findings and independently source-recheck affected claims before formal re-review. Record both reviewers' dispositions for the changed version before privacy clearance or release; older gate records do not automatically carry forward.
              • Record privacy clearance for that same exact artifact/evidence version, including exclusion of protected data and private-repository content from any public release; clearance for an earlier version does not clear the changed version.
              • Only after every evidence, review and privacy gate passes for that same exact artifact/evidence version, Bob and Ken record the named human release decision identifying that version before anything is made public; a changed or blocked artifact stays private and draft until its applicable gates and version-bound decision are complete.
              objectives
              • id
                500.P-a
                text
                Prepare and pin the exact artifact/evidence version for a paper-reproduction, governance-regime or red-team artifact using the common rubric, route-specific checks and pre-submission checklist. For an inaccessible paper reproduction, define a stranger-runnable substitute supporting only a narrower claim; without one, document the limitation and keep the work private and draft.
                misconception
                A documented access limitation satisfies the independent-rerun gate without anyone independently running a check.
                quiz item
                Given a full-paper access barrier, distinguish a version-pinned subset or released-artifact check that someone else actually reran for a narrower claim from a limitation-only record that leaves the rerun gate unmet; the artifact itself is human graded.
              • id
                500.P-b
                text
                Record each out-of-class gate against the exact artifact/evidence version in order from independent rerun through red-team ruling, accepted-finding closure/retests and source re-check, two-human review, privacy clearance and Bob and Ken's release decision. After a material change, reopen affected independent reruns and red-team rulings before formal re-review; publish only the version for which all gates pass and the release decision is recorded.
                misconception
                Reruns, rulings, reviews, privacy clearance or release approval for an older version automatically authorise publication after a material change.
                quiz item
                Given a changed claim, baseline or evidence after review, select the affected independent reruns and red-team rulings to reopen before formal re-review, then identify missing same-version source re-check, reviewer dispositions, privacy clearance or Bob and Ken's release decision that keeps the changed artifact private.
              starting sources
              • AiGovOps Foundation (2026). Level 400 · Prove it — worksheet — https://practice.aigovops-foundation.com/levels/400-prove/WORKSHEET.html (500.P-s1).
              • AiGovOps Foundation (2026). Study AI Like a Top-University Student, Free — https://www.aigovops-foundation.com/learning/university-ai-courses/ (500.P-s2).
              • Joelle Pineau et al. (2021). Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) — https://jmlr.org/papers/v22/20-303.html (500.P-s3).
              • Lipton, Zachary C. et al. (2019). Troubling Trends in Machine Learning Scholarship; ACM Queue 17(1), February 2019 (commentary; not a peer-reviewed research paper) — https://api.crossref.org/works/10.1145/3317287.3328534 (500.P-s4).
              • Raji, Inioluwa Deborah et al. (2020). Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing — https://arxiv.org/abs/2001.00973 (500.P-s5).
              • Lipton, Zachary C. et al. (2018). Troubling Trends in Machine Learning Scholarship; preprint (not peer-reviewed) — https://arxiv.org/abs/1807.03341 (500.P-s6).