Level 100: How AI works: AI at work and in life · Module 100.1

Try each exercise first, then open its model answer.

What a model is, and isn't · Workbook

Three exercises, one for each thing this module teaches. Everything here is set at Big Widget Company, a made-up company. The people, tools and answers are made up too. Use only what the lesson taught. You do not need to look anything up.

Exercise 1 (100.1-a)

At lunch, your coworker Sam says: "The AI tool is great. When I ask it something, it just looks the answer up in its big database."

Write three to five short sentences back to Sam. Your reply should:

  1. Say what a language model like the ones in this lesson (GPT-3, for example) does when it answers, in your own words.
  2. Say what the model can use besides what it learned in training.
  3. Say how a tool that searches documents first is different from the model on its own.
Model answer

A good reply covers all three points, in plain words. For example:

"It doesn't look the answer up. The kind of model in our lesson learned patterns from a huge number of examples. When you ask it something, it writes a new answer one small piece at a time, picking a likely next piece. It can also use the words you give it right now, like notes you paste in, without being changed or retrained. Some tools search a set of documents first and hand what they find to the model. That search is an extra step built around the model. The model on its own doesn't search."

What to look for:

  • Patterns, not a list of answers. The reply says the model writes a likely answer from patterns it learned. A reply that says it "finds", "fetches" or "looks up" the answer has kept the misconception. The reply does not need to say that every AI model works this way; this lesson studies the kind that writes one piece at a time.
  • The words in front of it. The reply says the model can use what you give it now, as well as what it learned.
  • Search is a separate step. The reply does not say every AI model searches, or that a search step means the answer is always right.

Exercise 2 (100.1-b)

Priya in quality asks an AI tool: "How long should the paint on a widget door dry before it is packed?" The tool answers, with no "maybe" or "I think":

"Paint on widget doors must dry for exactly 6 hours before packing. This is required by Big Widget Quality Rule QR-118, section 4."

(This answer and the rule number are made up for this exercise. Treat them as unchecked.)

Answer these:

  1. What does the sure tone tell Priya about whether the answer is true?
  2. The answer names a rule and a section. Does naming a rule show that the answer is true, or that anyone has checked it? Why or why not?
  3. Name two checks Priya could make that do not rely on the AI's answer.
  4. Priya asks the same tool again and gets the same answer. Is that a check?
  5. Priya cannot reach the rule book or the right person today. What should she write next to the answer in her notes?
Model answer
  1. Nothing that settles it. A model can write false things in a calm, sure voice. Tone tells you how the text was written, not whether it is true.
  2. No. A model can make up the support for an answer, such as a fake source or a fake reason. A named rule is something to check, not a check. It only counts once someone opens the real rule and finds it says this.
  3. Any two checks outside the answer, for example: open Big Widget's real quality rules and look for QR-118, section 4, and read what it says; ask the person in quality who owns the drying time; look at the paint maker's instructions the plant already uses.
  4. No. Asking the same model again is not a check. A check uses something other than the answer itself: the original document, a measurement, or a person who knows.
  5. "Not checked." That is an honest label. Then she finds the person who can help her check it, and does not pack widgets on the AI's word.

What to look for: the learner does not treat the sure tone, the rule number, or a repeat answer as proof. Each check they name goes outside the AI's answer.

Exercise 3 (100.1-c)

Assume each task uses an AI tool that Big Widget has approved for that kind of work, and that the information is allowed to go into it.

Sort each task into one of two columns: AI can help, if a person checks it or AI should not do this alone. For each task, write one short reason using the two questions: If the answer is wrong, how bad is that? and Can a person check it before anyone relies on it?

TaskColumnReason
A. Ideas for a name for the new team newsletter
B. Setting the pressure limit on a paint-shop machine
C. A first draft of a report that you will read and correct
D. Deciding which worker gets a pay rise
E. Rewording your own team update in simpler words, which you then read
F. A price quote sent straight to a dealer

Then answer:

  1. For each task in the "should not do this alone" column, name the role of the person who should say go ahead. (Make one up if you need to, such as "maintenance lead".)
  2. Now drop the assumption at the top. Your teammate wants to paste this month's unpublished sales figures into a free AI website the company has not approved. They promise to check every figure afterwards. Is that good AI? Why or why not?
Model answer
TaskColumnReason
A. Newsletter name ideasAI can help, if a person checks itA bad idea costs little, and you choose from the ideas yourself.
B. Machine pressure limitAI should not do this aloneA wrong setting is a safety risk. A person decides and owns it.
C. First draft of a reportAI can help, if a person checks itYou read and correct it before anyone relies on it.
D. Who gets a pay riseAI should not do this aloneIt is a decision about a person. A person decides and owns it.
E. Rewording your own team updateAI can help, if a person checks itYou wrote the facts, and you read the new wording before you send it.
F. Price quote straight to a dealerAI should not do this aloneA wrong price reaches a customer before anyone checks it.
  1. For example: B, the maintenance lead or safety lead; D, the worker's manager with HR; F, the sales manager who owns pricing. Any answer that names a role is fine. "A human in the loop" is not enough if nobody knows who can say go ahead.
  2. No. Checking the figures does not make it OK to give a tool information it is not allowed to see. Good AI needs both: the tool and the information are allowed, and a person checks the result. Module 100.7 covers what information may go into which tools.

What to look for: the left column is not "safe", because the person checking still owns the result. The right column is not "AI can never be used", because AI can still help a person prepare. The learner does not put a task on the left just because the AI can write it well.

Next module: 100.2 How it learned