Try each exercise first, then open its model answer.
Operating a model: monitoring, drift and rollback · Workbook
Three exercises, one for each thing this module teaches. Everything here is made up, set at Big Widget Company: the system, the people, the release names and every number. Do not use real names, systems or records from your own work. Write short answers. "Not checked" or "I don't know yet" is an honest answer if you also say who you would ask. A finished workbook describes your practice; it is not an assurance that any system is safe.
The exercises use a second made-up system, so you apply the module rather than repeat its record.
The made-up system. Big Widget's scrap predictor scores each batch of widget housings as it leaves the casting line: how likely is it to fail final inspection? The plant manager uses the score to pull high-risk batches for an early check. Final inspection takes three weeks, so the true answer for each batch (its label) arrives three weeks after the score was used. The current release is S7: model sp-7, data snapshot ds-2026-07, feature pipeline fe-7 (which reads the furnace's temperature sensors), serving build sv-7 and configuration cfg-7. Its Level 300 hold card names Priya Shah, quality-engineering lead, as holder. The card's stop: if the predictor is held, every batch gets the early check by hand.
Exercise 1 (400.4-a)
Monitoring that works before the labels arrive. Next month the plant replaces the furnace's temperature sensors with a new model that reports in a slightly different range.
- Why can an accuracy alarm alone not protect the plant in the first three weeks after the change?
- Write the three layers of the monitoring plan (inputs, outputs, outcomes). For each, say what it watches, whether it needs labels, and the one threshold that pages the holder.
- Who is paged, and what happens to everything that does not reach a paging threshold?
- An input-shift alert fires on the first day with the new sensors. What does it tell Priya, and what does it not tell her?
- Where do the monitor's settings live, and what happens to them if S7 is rolled back?
Model answer
- Accuracy needs labels, and labels here arrive three weeks late. Breck and colleagues (2017) warn that "it is not always possible to know the correct labels even shortly after serving time", and engineers in Shankar's 2022 interviews (a preprint) said labels "often arrived after a delay". For three weeks an accuracy alarm would be silent whatever happened. Tests passed before release do not help either: Sculley and colleagues (2015) say that "in the face of a changing world such tests are not sufficient".
A good plan has one paging threshold per layer. The numbers are yours to choose and justify; these are examples:
- Inputs (no labels). A schema check on each sensor reading (type, allowed range) and a daily shift test comparing each input's distribution with sp-7's training reference. Page: the shift test fails on two daily runs in a row, or any batch has readings outside the schema's range. One failed run opens a ticket.
- Outputs (no labels). The share of batches scored high-risk, overall and by casting line, compared with that line's share at launch. This comparison is this course's label-free proxy, not a check the sources define, and it is not prediction bias: it only says the scores have moved. Add an action limit on how many batches the predictor can pull in a day; Sculley and colleagues say that when such a limit is hit, "automated alerts should fire and trigger manual intervention or investigation". Page: any line's high-risk share is more than double its launch level, or the action limit is hit. The doubling is a limit Priya chooses, not the result of a statistical test.
- Outcomes (labels, three weeks late). When inspection results arrive, compare the predictor's error, overall and by line, with the bounds set at launch. Check prediction bias here too: by line, does the predicted failure rate match the failure rate inspection actually found? Sculley and colleagues' prediction bias compares "the distribution of predicted labels" with "the distribution of observed labels", so it has to wait for the labels. Page: any line's error or bias is outside the launch bounds.
Keep the two output checks apart. Two weeks of scores can look identical today and still turn out very differently once inspection is done. A steady output share does not show the scores are good.
Before you trust any of these thresholds, replay them over calm past periods and count how often they would have paged. If nobody has done that yet, the honest answer to "how often will this page falsely?" is "not known yet", with Priya as the owner of finding out.
- Priya Shah, the holder on the 300 card, is paged; the on-call engineer gets a copy. Everything else goes to a ticket. Shankar's interviewees said a surplus of false-positive alerts "led to fatigue and silencing of alerts". Because a page can still be missed, the card's stop does not wait for Priya: if the predictor is held, every batch gets the early check by hand.
- It tells her to look now: the inputs no longer look like the training data. It does not tell her the predictor is wrong. Rabanser and colleagues (2019) note that distributions "shift constantly, and often these changes are benign", and without labels the error on the new data cannot be computed directly. A good answer also notes that a sensor swap is a change in the source system, which Breck and colleagues warn "can radically change the feature's meaning". Priya should confirm the new sensors' units and range with the team that installed them.
- In this course's practice, the monitor's settings (reference data version, inputs watched, thresholds, who is paged) are part of the release's configuration, checked into a repository and carrying S7's release ID. If S7 is rolled back to S6, the monitor rolls back to S6's settings too, so it compares live data with S6's reference. That is the course's practice built on Sculley's configuration advice, not a rule from the sources.
Exercise 2 (400.4-b)
A release and rollback plan. Release S8 brings a new model (sp-8) and a new feature pipeline (fe-8) that adds the new sensors' humidity reading as an input. The draft plan says:
"Deploy sp-8 to all lines on Monday. If anything goes wrong, the on-call engineer swaps the model back to sp-7."
- List the four things the plan must pin together, with S8's version for each, under one release ID.
- What goes wrong if the on-call engineer swaps only the model back to sp-7?
- Rewrite the rollout as stages. For each stage, say what it can show, what it cannot show while the three-week labels are still pending, and who decides that it moves on.
- Name who can stop each stage and who can order a rollback.
- How will you show the rollback works before you need it?
- A rollback restores the system. What does it not undo, what does it not fix, and what should the plan record?
- After a rollback, when can Priya say the predictor is back at yes?
- Before Priya is asked to say yes to S8's first stage, what should the plan's author be able to say about it?
Model answer
One release ID, S8, pins:
- Model: sp-8, and the model specification and exact code that produced it.
- Data: the training-data snapshot (a new one, for example ds-2026-10), and frozen copies of any input owned by another team, such as the sensor feed.
- Code: feature pipeline fe-8 and the serving build (sv-8).
- Configuration: thresholds, the feature list, the monitor settings and the paging rules (cfg-8).
A rollback restores a release ID (S7 as a whole), never one part. The single release ID is this course's rule, built on the papers' reasons (Sculley's "Changing Anything Changes Everything" and Breck's call to revert "to a previous known-good state"); the papers did not test it.
- fe-8 keeps sending the humidity input, which sp-7 was never built for. At best the serving check rejects the requests and scoring stops, as when the made-up R13 rollback swapped only the weights; Breck and colleagues give the general case, where a server "will refuse to load the model". At worst the mismatch does not change the shape of the inputs and the old model runs silently on features it was not trained on. Sculley and colleagues' CACE principle, "Changing Anything Changes Everything", is why: rolling back one part leaves a combination nobody tested. Rolling back a model is not the same as rolling back code.
A good answer stages it, for example:
- Before release: an automated check that must "either bless the model or veto it", comparing sp-8 with sp-7 slice by slice with tighter thresholds (Breck, Infra 4 and Model 6).
- Shadow (one week): sp-8 scores every batch, but its scores are not used. Can show: the release loads, serves, passes its input checks and gives plausible scores. Cannot show: what happens when the plant acts on sp-8's scores. Pulling and reworking a batch changes its inspection result (fact A in Exercise 3), and Shankar's interviewees note it is "impossible to evaluate the model in shadow mode because users do not interact with shadow predictions".
- One casting line (one week): sp-8's scores drive that line; the others stay on S7. Can show: early signals (inputs, output share, action limit) for that line, with fewer batches exposed if sp-8 is bad. Cannot show: accuracy. Inspection takes three weeks, so after one week of shadow and one week on a line, not one S8 label has come back.
- All lines. Breck and colleagues recommend running "old and new models concurrently, with new models only seeing a small fraction of traffic, gradually increased as the new model is observed to behave sanely".
Who moves it on: each move is Priya's gate decision, not an automatic step. Her answer is one of four: allow (go ahead), constrain (only this far, for example stay on one line), hold (wait for a person or for labels) or deny (no). The record says who answered, when, and what the answer rested on. Neither the predictor nor the monitor ever gives that answer. "No page yet" is not evidence that sp-8's scores are right. A good answer states what is still not known (outcome quality) and when it will be (the first inspection results, three weeks after the line stage began). How much exposure to allow before labels exist is Priya's call to make and record; this workbook does not set it for her. When the labels do arrive, the batches sp-8 pulled for rework are not clean evidence of its accuracy, because its own action changed their results; the random never-pulled sample from Exercise 3 is the cleaner check.
- Priya Shah, as holder, can stop any stage and order a rollback. The on-call engineer may roll back to S7 without waiting, but must tell Priya at once. NIST's framework asks that responsibilities to "supersede, disengage, or deactivate" an AI system be "assigned and understood" (MANAGE 2.4).
- Practise it: a dry rollback in a sandbox, restoring S7 as a whole and checking that scoring resumes and the monitor compares against S7's reference. Breck and colleagues: operators "should practice doing it normally, when not in emergency conditions".
- It does not undo what S8 did while it ran: batches it scored, batches pulled or not pulled for an early check. The plan (and the incident note, if it happens) records which batches S8 scored, so they can be marked before the next training run. That is the course's practice. It also does not make S7 right for a world that has changed. If the new sensors from Exercise 1 are already installed, S7 was trained on the old sensors' readings and has never seen the new range either. Restoring S7 pins what it learned from; it still reads today's inputs. So Priya may keep a hold after the rollback, for example the early check by hand for batches the new sensors scored unusually, until S7 is shown to be inside its bounds on the new readings. That hold needs an owner (Priya) and a review date, so it does not quietly become permanent: for example, when the first inspection results for batches scored on the new sensors land, three weeks after the swap.
- Not when the rollback finishes. Stopping a bad release is not, by itself, a return to yes. Priya can call it recovered when the labels (or another check she has agreed in advance) show the restored release is inside its launch bounds for the lines that failed, and she agrees. Until then the honest status is "not known yet", with Priya as owner. The incident record, the list of affected batches and the batch-level evidence stay in the team's access-controlled records. A receipt of her decision holds no private content: the decision, who made it, when, the release ID and references (such as hashes) to the evidence. In this module it can be an unsigned decision record. A hash in it lets someone confirm the referenced record has not changed; it does not show the record is right, and it is not permission to publish what the record contains.
- Before Priya is asked to say yes to S8's first stage, the plan's author writes "I" statements about it, or says plainly which they cannot make yet and who will close the gap. A good answer covers: which threshold pages Priya on each layer and which threshold on the 300 card it measures; the combined rule's false-page rate from a replay, or "not known yet" with an owner; that S7 can be restored as a whole and the rollback has been rehearsed; that every consumer of the scores (including the maintenance dashboard from Exercise 3) is named in the release plan; and who answers allow, constrain, hold or deny at each stage, on what evidence. The statements are evidence for Priya's decision, not the decision itself.
Exercise 3 (400.4-c)
Hidden technical debt. Three facts about the scrap predictor (all made up):
- A. Batches the predictor scores as high-risk are pulled and reworked before final inspection, so most of them pass. Those inspection results become the next training labels.
- B. The maintenance team's dashboard reads the predictor's score log to decide when to service the furnace. Nobody told the predictor's team.
- C. In S8, adding the humidity input also changed how much the model relied on furnace temperature, though nobody changed the temperature input.
- Name the kind of hidden debt in each of A, B and C.
- Name one control for each.
- A manager says: "The model is the system; get the model right and the rest will follow." Answer them in two or three sentences, citing the source.
Model answer
- A is a direct feedback loop: the predictor influences "the selection of its own future training data" (Sculley and colleagues). Rework turns its high-risk batches into passes, so it may learn that such batches are fine. The loop runs through the plant, not through code, so it is hard to see, and harder still "if [it occurs] gradually over time".
- B is an undeclared consumer: another system is "silently using the output of a given model as an input", which creates "a hidden tight coupling". When S8 changes the scores, the furnace's service schedule moves and nobody knows why.
- C is entanglement: "Changing Anything Changes Everything". Adding one input changed "the importance, weights, or use of the remaining" inputs. "No inputs are ever really independent."
- A is a direct feedback loop: the predictor influences "the selection of its own future training data" (Sculley and colleagues). Rework turns its high-risk batches into passes, so it may learn that such batches are fine. The loop runs through the plant, not through code, so it is hard to see, and harder still "if [it occurs] gradually over time".
- A: keep some data out of the loop. For direct loops like this one, Sculley and colleagues suggest "some amount of randomization" or "isolating certain parts of data from being influenced by a given model". The course's way of doing that here: a randomly chosen sample of batches that are never pulled early, whatever their score. Split that sample in two. One part supplies training labels the predictor did not cause; the other is kept only for checking the model, never for training it. Being outside the loop does not, by itself, make data a fair test set. How large the sample is, given that those batches skip the early check, is Priya's decision to make and record.
- B: close the door and keep a list: logs readable only by declared consumers, each named in the release plan and told before a release. Sculley and colleagues point to "access restrictions or strict service-level agreements (SLAs)"; Breck and colleagues give consumers the matching duty to "make sure that the dependent team knows your team is using the data".
- C: before release, compare the new version's predictions with the old one's slice by slice (by casting line, by furnace), with a threshold "compared to the previously released model" (Breck, Model 6).
- A: keep some data out of the loop. For direct loops like this one, Sculley and colleagues suggest "some amount of randomization" or "isolating certain parts of data from being influenced by a given model". The course's way of doing that here: a randomly chosen sample of batches that are never pulled early, whatever their score. Split that sample in two. One part supplies training labels the predictor did not cause; the other is kept only for checking the model, never for training it. Being outside the loop does not, by itself, make data a fair test set. How large the sample is, given that those batches skip the early check, is Priya's decision to make and record.
- Sculley and colleagues (2015) found that "Only a small fraction of real-world ML systems is composed of the ML code", and suggested a mature system might be "(at most) 5% machine learning code and (at least) 95% glue code", an estimate from their experience, not a measurement of every system. The debt in A, B and C sits in the data, the plant and the other teams around the model, so that is where the controls go too.