Level 400: Engineering AI systems · Module 400.R

Five coworkers from different jobs compare notebooks with different coloured covers.
Does it still work?

Engineering AI in your role (role tracks) · Plain gear

As of 19 November 2026. Reviewed and published, version 1.0.0. Big Widget Company and its people, models and numbers are made up. The real papers, framework and court case are cited. The three questions, the tracks and the rails are this course's own practice, not a law. Where we do not know something, we say so.

Three questions and a badge

Every role track asks of one model: does it work (evaluation, Get to Yes), does it still work (monitoring, Stay at Yes), and could a stranger check it (evidence, Recover to Yes).
The three questions are the course's track structure. Monitoring of functionality and behaviour follows the AI RMF (s1); short records that travel with a model follow model cards (s2).
Figure data as text

Three questions for any role

StageWhat it shows
GET TO YES (COURSE)Does it work? Held-out data like real use, a baseline, uncertainty, by slice.
STAY AT YES [s1]Does it still work? Which signal, which threshold, who is told, and hold or roll back.
RECOVER TO YES [s2]Could a stranger check it? Which records, which versions, decided by whom, checkable without trust.

Priya is an engineer at Big Widget Company. Big Widget is a made-up company with 100,000 people. It makes big, costly widgets that are a bit like cars.

Priya's team built a camera model for the paint line. It looks at every painted panel. When it sees a flaw, it flags the panel so a person can look. It never throws a panel away on its own.

Her manager asks three questions:

  1. Does it work?
  2. Does it still work?
  3. Could a stranger check?

You met the three yeses in Level 100. These questions fit them. "Does it work?" comes before launch: that is Get to Yes. "Does it still work?" is Stay at Yes. When something goes wrong, you stop the harm, find the cause, fix it and test again. You show a stranger what happened. Only then do you say yes again. That is how you Recover to Yes.

Priya also has a new certificate from an exam. She is proud of it. Her manager says, "Great. Now show me the model's records." This module is about why both matter, and why they are not the same thing.

You will pick the track for your own role. Then you will answer the three questions for one model in your job. You will also learn what a certificate shows about you, and what it does not show about your model.

Question 1: Does it work?

Think of a new driver. Passing a test on an empty car park does not show they can drive in town. The test has to look like the real road.

So Priya tests the model on photos from the real line. They come from the same cameras and shifts, and from weeks the team did not use while building it.

Even that is not the whole story. One group of researchers rebuilt a famous photo test. They followed the original steps closely.1 The models still scored lower on the new photos, by 11 to 14 points out of 100.1 So one test is never the last word. Priya writes down exactly how her test was made.

She also does four more things:

If the results are too close to call, the answer is "not yet". The model waits, and the team tests more. "Not yet" is not a pass.

Question 2: Does it still work?

A smoke alarm is not checked once and forgotten. You test it, and it keeps watching.

A model is the same. The checklist says test data is always older than real use. So a score before launch is only a guess about later.2

Priya sets up three alarms:

The sample of passed panels tells Priya how many flaws slipped past. Ask "out of how many?" To know what share of all flaws the model catches, she also needs the flaws it did flag, counted over the same weeks.

Switching back can help when the new version caused the problem. It can also help after a change in the world, but only if someone shows the old version is still safe today. Say the cameras change. The new photos might make the old version a little worse but still good enough. Or they might hide flaws from both versions, and then switching back fixes nothing. Knowing the cause does not tell you which. A check on today's photos does. So first, people check every panel by hand. They keep doing it until a named person has checked that the old version works well enough on today's photos.

Switching back is also not proof the problem is fixed. The team still finds the cause and tests again before saying yes a second time. Until then, the honest answer is "on hold", not "fixed".

Some jobs find out the truth late. A worn part may fail months after the model's warning. A forecast is only right or wrong when the month is over. Then the alarm has to wait for the truth. Say how long the wait is, and who checks in the meantime.

A US government framework for AI risk says the same thing in its own words. It asks that AI systems are watched when in use, with plans for problems, fixes and changes.4

Question 3: Could a stranger check?

Picture a stranger who has never met Priya's team. Could they check the model's story without just taking the team's word for it?

Researchers suggested a "model card". It is a short fact sheet that travels with a model. It says what the model is for, how it was tested, and how it did for different groups.3 Priya writes one.

But the same researchers warn that a card is only as honest as the people who wrote it.3 They suggest outside checks too.3 A signature on the card shows who wrote it and that nobody changed it later. It does not show the card is true. So the stranger also needs the test records behind it, and the steps to run the test again.

The card goes into a signed bundle with the test records and the version numbers. The bundle shows if anyone changed it. It holds fingerprints of the files, the versions and the decisions. It does not hold private things: no customer details, no staff details, no raw outputs. Those stay in Big Widget's own locked work records. A fingerprint points to a file. It is not permission to share the file.

Someone who did not build the model decides on the release: go ahead, only this far, wait for a person, or no. "Wait for a person" is not a slow yes. A named person ends the wait, or the answer stays "wait". An AI tool can help write the records, but it never makes this decision. The US framework says the people who build a model should be kept apart from the people who check it.4 Having that checker sign the release is this course's own rule (400.9). A stranger can then check the bundle on their own, against a key and a checkpoint they got another way (you met this in 400.8).

Why this is Priya's job, not just the rules team's job. Priya picked the test photos. She set the line. She chose the alarms. Only she can write those choices down while she makes them. The US framework says roles for this work should be written down and clear to everyone.4

What goes wrong without it. In England, a judge looked at the Post Office's computer system, called Horizon. In December 2019 he found that the older version "was not robust".5 Then came Horizon Online, in a form called HNG-X. He found it better, but said it still did not justify the trust the Post Office placed in its accuracy.5 He said a later version, HNG-A, was a different matter: experts agreed it was far more robust.5 Horizon was an accounts system, not AI. The lesson still fits: being sure is not the same as having proof.

The eleven tracks

Each track is one kind of engineering job at Big Widget. The examples are made up.

TrackOne model (made up)Does it work?Does it still work?What a stranger needs
ML engineeringA chat helper trained more on service manualsRe-run the safety test after every change. Researchers showed extra training can weaken safety rules.6Fixed test questions, every releaseWhich versions; both test runs
Data scienceA model that sorts warranty claimsTest on later claims; beat today's ruleA person checks a sampleFact sheet with results by group
Data engineeringThe claim-sorting model, through the pipeline that feeds itYour part: check every load. The model's test comes from the data science teamAlarm when a data source changes2Which data went into each model version
MLOps and platformThe paint-line camera model, through the system that runs itYour part: try it on a little traffic first.2 The model's test comes from the inspection teamAge, speed and quality alarmsProof that switching back works
Security engineeringThe chat helperTry to make it leak what it learned. Researchers pulled hundreds of exact passages out of one model.7Watch for odd questionsAttack test results; locked logs
Evaluation and qualityThe claim-sorting model, through the company's test kitFresh tests, made the written-down way.1 Your part: check the kit gets a known answer rightRe-test after every changeTests and tools written down4
Manufacturing and quality inspectionThe paint-line camera modelReal line photos, split by colour and cameraSample of passed panelsFact sheet, versions, signed release
Maintenance and reliabilityA model that warns before a part wears outTest on machines it never saw; beat the fixed scheduleCount failures it missed, once the truth comes inWhich machines and dates it was tested on
Forecasting and supply chainA parts-demand forecastSmaller error than a simple forecast on later monthsTrack the error each month, after the month endsWhich data each forecast used
Service and warrantyA model that sends safety claims to engineersHow many real safety claims it sendsA person checks claims it did not sendResults by product line
Engineering and simulationA fast stand-in for a slow stress testSmaller error than allowed, compared with the full testSpot checks; alarm outside the tested rangeThe range it was tested on4

Three tracks look after a part, not a model. Data engineering, MLOps and evaluation work on a pipeline, a platform or a test kit. Those tracks still pick one model the part supports. They test their own part. They also name the model's own test, and the team that owns it. A clean pipeline or a safe roll-out does not show the model is good. A test kit cannot prove itself right by existing.

Your certificate and your model's records

Level 400 has four study "rails". Each one helps you prepare for a kind of outside exam. The course does not name any exam or exam company yet. Bob has to clear each name first.

Each rail draws on lessons in this level. The codes, such as 400.6-a, are the lesson goals you can look up.

RailUsually taken byLessons it draws on
ML engineeringML engineering, MLOps and platform, inspection, maintenance, simulation400.2-a choose between asking, looking things up and extra training · 400.2-c re-test safety after extra training · 400.L-a the hands-on extra-training lab · 400.4-a watch a model in use, even when the truth comes late · 400.4-b plan a release and a switch back · 400.6-a design a test that looks like real use
DataData science, data engineering, evaluation and quality, forecasting, service and warranty400.3-a write a data fact sheet and a model card · 400.3-b trace where data came from, and its licence · 400.6-a design a test that looks like real use · 400.6-c report a result with how sure you are · 400.7-a check results group by group
AI securitySecurity engineering400.5-a name the kinds of attack · 400.5-b stop a model leaking what it learned · 400.5-c put limits round a model that reads untrusted text · 400.8-b build a record that shows if it was changed
AI auditAnyone who will check other teams' models400.8-a why a computer's output is not proof on its own · 400.8-c what a bundle must hold, and must never hold · 400.9-a check the idea behind a rule before it runs · 400.9-b write the gates as rules · 400.9-c check a stranger's bundle offline

Priya works in inspection, so she takes the ML engineering rail. She picks two of its lessons. 400.6-a matches her test on real line photos. 400.4-a matches her alarms.

Now back to Priya's certificate. A driving licence shows you passed a test. It does not show the car you are driving has working brakes. You need the car's service record for that.

A certificateA model's records
AboutA personOne model, one version
ShowsThe person passed an exam on a dateHow the model was tested and watched, and who said yes
Does not showThat any model they built is safeWhether the team holds any certificate

The checklist from earlier gives points to a system, not a person. A test earns points only if it is actually run. Run by hand, with the results written down and shared, it earns half a point. Run automatically by a system, again and again, it earns a full point.2 So a test nobody ran scores zero, however good the team's certificates are. This course also asks for the results of automatic tests to go in the bundle. That is our own rule, not the checklist's.

A certificate is a good reason to trust that someone can make good records. It never replaces the records. A level or badge describes a person's practice, never a system's safety.

Your turn

  1. Pick your track, or write a new row for your job.
  2. Pick one model. Say in one sentence what it is for, and what it must never do alone. If your track looks after a part, name the model the part supports.
  3. Does it work? What real-life test data? What do you compare it against? Is a higher or a lower number better? Which groups do you check on their own?
  4. Does it still work? What alarm? Who gets told? What happens next, and what waits while they check? How long until the truth comes in?
  5. Could a stranger check? What records? Who says yes, if not you? What stays out of the bundle?
  6. Your rail. Name your rail. Pick two lessons from its row in the table. For each, write one sentence on how it helps with your model. Then finish this sentence: "My certificate would not show that my model ..."

You will use your answers in the project (400.P). Plain-gear learners use prepared outputs and do not need to write code. If you do not know an answer yet, write "I don't know yet" and name who you will ask.

What this lesson does not say

Sources

  1. s4 Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar (2019). Do ImageNet Classifiers Generalize to ImageNet?. Paper. Checked 2026-11-19 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 3 re-check, 19 Nov 2026. ↩ ↩ ↩
  2. s3 Eric Breck, Shanqing Cai, Eric Nielsen, Michael Salib, and D. Sculley (2017). The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction. Paper. Checked 2026-11-20 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 4 re-check, 20 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩ ↩
  3. s2 Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, et al. (2018). Model Cards for Model Reporting. Paper. Checked 2026-11-19 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩
  4. s1 Elham Tabassi (National Institute of Standards and Technology) (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. Standard. Checked 2026-11-19 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩ ↩ ↩
  5. s7 Mr Justice Fraser, High Court of Justice, Queen's Bench Division (2019). Bates v Post Office Ltd (No 6) "Horizon Issues" [2019] EWHC 3408 (QB). Record. Checked 2026-11-19 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 3 re-check, 19 Nov 2026. ↩ ↩ ↩ ↩ ↩
  6. s5 Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, et al. (2023). Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!. Paper. Checked 2026-11-19 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 3 re-check, 19 Nov 2026. ↩ ↩
  7. s6 Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, et al. (2021). Extracting Training Data from Large Language Models. Paper. Checked 2026-11-19 by source-checker agent (Claude, fresh session separate from the writer, reviser, item writer and earlier checkers), 400.R round 3 re-check, 19 Nov 2026. ↩

Next: the quiz