Level 100: How AI works: AI at work and in life · Module 100.2

Try each exercise first, then open its model answer.

How it learned · Workbook

Two exercises, one for each objective. Both ask you to explain how a model learned and what that means. Neither asks you to train a model. Use only what the lesson taught. If you are not sure of a fact, write "not checked" instead of guessing.

Exercise 1 (100.2-a)

A colleague at Big Widget Company (the made-up company used in this course) says: "Someone at the AI company programmed in every answer the chat model gives."

Write a short reply, five to eight sentences, in plain words. Your reply should:

  1. Say what the model's job was in the first big step of training.
  2. Say what "training" means: what happens when the model's guess is wrong.
  3. Give one number from the lesson that shows how large the training was, with the model's name and year.
  4. Describe the second big step: what people did, and what the model was adjusted toward.
  5. Name one thing the tuned model still got wrong, so your colleague does not think tuning made it perfect.
Model answer

Nobody typed in the answers. In the first step, the model read a huge amount of text, and its only job was to guess what comes next. Each time its guess was off, it was adjusted a tiny bit, so that its guesses matched the text a little better. This happened at a huge scale: GPT-3, described in 2020, was trained on 300 billion tokens (small pieces of text). A model that only guesses the next word is not yet a good helper, so people then tuned it. In the InstructGPT study, about 40 hired people wrote example answers, then compared the model's answers and ranked them, and the model was adjusted toward the answers they preferred. People liked the tuned model's answers better than those of a model 100 times bigger. But it still made simple mistakes and sometimes made up facts.

A good answer:

  • says the first job was predicting (guessing) the next piece of text, not storing answers (s3);
  • describes training as many small adjustments that make the predictions match the training text better (s2);
  • gives a correct, attributed scale figure, for example GPT-3 (2020): 175 billion parameters or 300 billion training tokens (s1), or PaLM (2022): 540 billion parameters, 780 billion tokens (s7);
  • names the human-feedback steps: people write examples, people compare and rank answers, the model is adjusted toward what they preferred (s3);
  • keeps one limit: the tuned model still made mistakes and made up facts (s3).

Watch for these errors:

  • "Tokens", "parameters" and "adjustments" are different things. 300 billion is a count of training tokens for GPT-3, not a count of adjustments.
  • Do not say people checked every fact during tuning. They compared answers and picked the ones they preferred.
  • Do not say this is how every AI model learns. The two-step recipe describes chat models like those in the lesson. GraphCast learned from examples but was not trained on text or tuned by people.

Exercise 2 (100.2-b)

Big Widget Company's planning team uses a model that learned from ten years of the company's widget sales. Next quarter the company plans to launch a brand-new kind of widget, unlike anything it has sold before. A manager wants the model's sales forecast for it, and says: "The model is a machine, so it is neutral and up to date. We can use its number as it is."

Write a short note to the manager, in plain words. In it:

  1. Explain, in one or two sentences, why the model's output reflects the data it learned from.
  2. Name the gap in this case, and what it means for the forecast.
  3. Name the cutoff issue: what the model does not know, and what could help.
  4. Explain why "it is a machine, so it is neutral" is wrong. Use one finding from the lesson about unfair patterns or about whose preferences a model reflects.
  5. Write this lesson's three data questions, one for each yes, for this forecast. Say who decides.

Use made-up details only. Do not put real sales figures or customer details in your note.

Model answer

The model learned only from what was in its training data: ten years of sales of widgets we already sell. Its answers reflect that data, including what is missing from it and when it ends.

The gap. The new widget is unlike anything in that data. The model has no comparable history to go on, but it can still give a confident number. A model tends to handle well what was common in its data and to handle badly what was rare or missing.

The cutoff. The model only learned from sales up to the date its data was collected. It knows nothing that happened after that, such as recent orders. A model can use newer facts if someone gives them to it, but if nobody does, it can sound sure and still be wrong. NIST calls confidently stated but false content "confabulation".

Not neutral. People choose what goes into training data and how much each part counts; the builders of The Pile, for example, gave more weight to the parts they judged higher quality. The builders of GPT-3 wrote that it "retains the biases of the data it has been trained on". Tuning with human feedback did not remove that: InstructGPT did not do significantly better than GPT-3 on two standard bias tests. Being a machine does not make a model neutral or up to date.

The data questions.

  • Get to Yes: What data did the model learn from, and when does it end? Does a brand-new widget look like anything in it? Here the answer is no, so the number is not reliable on its own. If we do not know the data's end date yet, we say "not known yet". The planning lead decides.
  • Stay at Yes: If we use the model for products we already sell, keep asking: has the task moved outside what it learned from? Has the market changed since its data ends?
  • Recover to Yes: If a forecast for a new product goes wrong, stop using the model for that kind of task. The planning lead owns the error and fixes the plan, including for anyone affected. The rule changes: forecasts for new products wait for a person. We only call it recovered once later forecasts show the new rule working.

These are only the data questions. Good answers to them are not, on their own, permission to act. Lesson 100.9 covers the full gate.

A good answer:

  • ties the forecast's weakness to the training data: what was in it, what was missing, and when it ends;
  • names the gap (no comparable product) and says a confident answer is not a correct one (s5);
  • names the cutoff, and says supplied newer facts can help but the model does not get them by itself (s1, s5);
  • uses a real finding from the lesson against "neutral": chosen data (s4), GPT-3's own statement about bias (s1), PaLM's stereotype finding (s7), tuning not fixing bias (s3), or tuning reflecting a specific group's preferences (s3);
  • keeps a person in charge: the planning lead decides and owns the plan;
  • keeps private data out of the note.

Watch for these errors:

  • Do not claim the model "can never" forecast anything new. The problem here is that it has nothing comparable to learn from, not that new things are impossible to predict.
  • Do not give a knowledge-cutoff date for a real model unless you have checked it. The lesson dates only one part of GPT-3's data (Common Crawl, 2016 to 2019), not the whole model.
  • Do not treat good answers to the data questions as a "go ahead". They are one part of the check.

Next module: 100.3 How it writes