Level 100: How AI works: AI at work and in life · Module 100.9

Try each exercise first, then open its model answer.

The three yeses: the pre-pend gate · Workbook

Three exercises, one for each thing this module teaches. Big Widget Company and everyone in it are made up. Do not use real names, customers, suppliers or documents from your own work. Write short answers. "I don't know yet" is an honest answer if you also say who you would ask.

Your answers are a private learning record. Do not paste customer, supplier or personal details into them, and do not post them anywhere to show your rule works.

Exercise 1 (100.9-a)

Who answers for the chatbot? Big Widget Company plans a chatbot on its website to answer customers' questions about delivery dates for new widgets. In the planning meeting, a colleague says:

"If the bot gets a date wrong, that's on the bot, not us. And every answer links to our real delivery policy page, so customers can always check."

  1. Using the Air Canada chatbot case, explain in two or three sentences why the colleague's first sentence is wrong.
  2. Does linking every answer to the correct policy page protect the company? Use the case to say why or why not.
  3. Name one thing the Air Canada case does not prove.
  4. What has to happen before the chatbot answers its first customer?
Model answer
  1. In Moffatt v. Air Canada (2024), Air Canada argued it could not be held liable for what its website chatbot said, as if the chatbot were responsible for itself. The tribunal called that a "remarkable submission". It held that the chatbot is part of Air Canada's website and that Air Canada is responsible for everything on its website, chatbot included, and it ordered Air Canada to pay Moffatt. A chatbot on Big Widget's website speaks for Big Widget. If the company's AI says it, the company said it.
  2. No, not by itself. Air Canada's chatbot linked to Air Canada's own bereavement-travel page, and that page said the opposite of what the chatbot told Moffatt. The company was still held to the answer its chatbot gave. A correct link next to a wrong answer does not cancel the wrong answer.
  3. Any one of these: it is one tribunal in one Canadian province deciding one small claim, so it is not a law for every country; it is not legal advice for Big Widget's own case; it does not say what every court would decide.
  4. A person decides what the chatbot may say, and a named person owns it, before it answers anyone. In this course that means a written rule with all six blanks filled (Exercise 2). The chatbot cannot approve itself.

Exercise 2 (100.9-b)

Write one rule, then test it. Dana works in accounts payable at Big Widget Company. Suppliers often email to ask when an invoice will be paid. Dana wants the company's approved AI tool to draft the replies, so they go out faster.

  1. Write one rule, in your own words, with all six blanks:

    This AI may [do what] for [whom] only when [condition]. Otherwise it [falls back to what]. [Named person] owns it. We keep [this receipt].

    Say who is allowed to ask for a reply, as well as who the reply is for.

  2. Say what checks the draft before it goes out, and why it cannot be the AI tool itself.
  3. Test your rule. For each case, say which of the four answers your rule gives (go ahead, only this far, wait for a person, no) and which blank decides it:
    • A. The draft gives a payment date that matches the invoice record, and Dana has checked it against the record.
    • B. The draft says "You'll be paid next week", but the invoice record shows no approved payment date.
    • C. The finance system is down, so nobody can check the date.
    • D. Someone in sales, not accounts payable, asks the tool to send a payment date to a supplier they know.
  4. List what goes in the receipt, and what stays out.
Model answer
  1. One good rule:

    This AI may send a reply it has drafted, telling a supplier when an invoice will be paid, for suppliers with an open invoice, when a member of the accounts payable team asks for it only when the payment date in the draft is copied from that invoice's record in the finance system, the draft adds no other promise, and an accounts payable team member has checked the date against the record. Otherwise the draft is not sent: the accounts payable team member writes the reply themselves, or tells the supplier they will come back with a date. The accounts payable lead owns it. We keep a receipt for each reply: which rule, which invoice record was checked, what the check found, the decision, who checked and the time, with no email text or supplier details in it.

    The action this rule decides is sending, because sending is what reaches the supplier. The tool may write a draft privately; the draft goes nowhere until the check passes. Your words can differ. Check that every blank is filled, the action is one action (not "use AI"), the owner is one named role or person who can release a hold, and a coworker could follow it without asking you what you meant. "Suppliers with an open invoice" is who the reply is for. "A member of the accounts payable team" is who may ask. Being a supplier, or knowing one, is not permission to start the action.

  2. A person who did not write the draft (here, the accounts payable team member) opens the invoice record and compares the date before the reply goes out. The AI tool cannot be the check: putting the rule only in the tool's instructions means the same system that might get the date wrong decides whether it is right. That is not a gate.
  3. Tests:
    • A: go ahead. "Only when" is met: the date comes from the record and a person checked it.
    • B: no, then wait for a person. "Only when" fails, because the date is not in the record. The draft is not sent. "Otherwise" decides what happens next: Dana writes the reply, or tells the supplier a date will follow.
    • C: wait for a person. The condition cannot be checked, so it is not met. Not knowing is not a yes. "Otherwise" applies, and the supplier is told a date will follow.
    • D: no. The person asking is not in accounts payable, so "for whom / who may ask" fails. The accounts payable lead can decide whether sales should ever be allowed to ask; the AI cannot.
  4. In the receipt: which rule, which invoice record (by its reference number), what the check found, the decision (sent, rewritten or held), who checked, and the time. Not in the receipt: the supplier's email, the draft text, contact details or bank details. If company rules allow the emails to be kept, they stay in the normal private finance records, seen only by the people who need them.

Exercise 3 (100.9-c)

Stay at yes, then recover to yes. Dana's rule (Exercise 2) has been running for four months. The drafts have always been right, so the team now sends them after a quick glance, without opening the invoice record. Last week a draft gave a supplier a payment date for a different invoice. The supplier planned around that date, and the money did not arrive.

  1. Which of the three Stay at Yes questions would have caught this, and when?
  2. Walk through the four Recover to Yes steps for this case. Be specific about who does what.
  3. Name one documented case from this module and say what it shows about staying or recovering, and what it does not show.
  4. Before switching the AI drafts back on, what evidence would you want that you are really back at yes?
Model answer
  1. "Am I still checking?" The rule said a person checks the date against the record, but the team drifted into skimming because the drafts had been right. NIST's Generative AI Profile warns that over time people may over-rely on AI, which it calls automation bias. Asked honestly at any point after the team started skimming, that question would have shown the check had stopped. "Is the receipt still being kept?" would also have helped: receipts saying "checked" when nobody opened the record are a warning sign.
  2. Recovery:
    • Stop. The accounts payable lead switches off AI drafts for payment-date replies. Every reply is written by a person until the lead says otherwise.
    • Own it. Big Widget, not the AI tool, tells the supplier the date was wrong and gives the right date. As in the Air Canada case, the company should not plan on blaming the tool.
    • Fix it. Use the receipts to find every reply sent since the team stopped opening the record, and check each date against its invoice record. Contact every supplier who got a wrong date, correct it, put right any harm where the company should, and record what was done for each, including anyone who could not be reached. The list of suppliers is the first step, not the fix.
    • Change the rule. Find which part of the gate failed. Here the rule was fine on paper, but the check was not being done. One change: the receipt must record the invoice number the checker opened, and the draft is not sent unless that number matches the invoice being asked about. Add a separate Stay at Yes check: the lead reviews a sample of receipts each month. Then go back through Get to Yes.
  3. One good answer: the Tesla Autopilot recall (2023). Tesla's report said the checks on whether a driver is paying attention may not be enough to prevent misuse in certain circumstances: a system that relied on people to keep watching. Tesla announced a voluntary recall with a free over-the-air update adding more alerts and checks, and a way to take Autosteer away from a driver who keeps failing to pay attention. It did this while disagreeing with the regulator's analysis; owning a repair does not require agreeing with every finding. What it does not show: the report describes a plan. It does not record when every car got the update, or whether it worked. An announced repair is not a proved return to yes.
  4. Evidence that the fix reached the people affected and that the changed rule works. For example: a record for each affected supplier showing they were contacted and what was done; a test of the repaired gate in which a draft with the wrong invoice's date is stopped; and the first weeks of receipts showing the invoice number recorded and checked every time. The accounts payable lead, not the AI tool, decides when to switch it back on.

Next module: 100.R AI in your job (role tracks)