Try each exercise first, then open its model answer.
Grounding it in your documents · Workbook
Three exercises, one for each thing this module teaches. Everything here is set at Big Widget Company, a made-up company. The people, tools, documents and answers are made up too. Use only what the lesson taught. You do not need to look anything up.
In these exercises, Priya owns Big Widget's warranty page and Tom runs the dealer assistant, the AI tool that answers dealers from Big Widget's documents.
Exercise 1 (200.3-a)
Big Widget's warranty page (made up) says, in section 1:
"Standard widgets carry a 12-month warranty from delivery. The warranty covers defects in parts and workmanship. It does not cover damage from misuse."
A dealer asks: "A standard widget we took delivery of 14 months ago has a faulty part. Is it covered?" Your coworker Sam gives the dealer assistant the warranty page and asks three times. Sam gets three answers:
- A. "No. Standard widgets carry a 12-month warranty from delivery."
- B. "No, but you can buy an extended warranty for 10% of the widget's price."
- C. "Yes. The warranty covers defects in parts."
Sam says: "I attached the warranty page, so all three came from it. I'll just use whichever one I like."
- For each answer, say whether it is supported by the page, only sounds like it, or is wrong, and why.
- In two or three sentences, tell Sam what grounding changes.
- Name three things grounding does not fix.
Model answer
1. The three answers.
- A is supported. The page gives 12 months from delivery, and 14 months is past that.
- B only sounds like it. "No" is right, but an extended warranty and a 10% price are not on the page. They sound like a warranty policy, which is why they are easy to miss.
- C is wrong. It takes one true part of the page (defects in parts are covered) and drops the 12-month limit.
2. What grounding changes. For example: "Giving the tool our warranty page means it can answer from our own rule instead of a guess about warranties in general. And when Priya updates the page, the tool can use the new version. That doesn't mean every answer came from the page: only A did."
3. What grounding does not fix. A good answer names all three:
- The model may not use all of what it is given. In a 2023 study, models did worse when the answer was in the middle of what they were given. Having the page is not the same as using it.
- The model can go beyond the document. It can quote the right rule and add something that is not in it, like the extended warranty in B. NIST calls confidently stated false content confabulation.
- The documents themselves can be wrong. If the page is out of date, or two pages disagree, a model that reads them faithfully will still give a wrong answer.
What to look for: a reply that says "it came from the page, so it's right", or lets Sam pick the answer they like, has kept the misconception. Do not mark a reply down for not giving the 2023 study's details; the point is that being given a page is not the same as using it.
Exercise 2 (200.3-b)
Part 1. Write the request you would give the dealer assistant so that its warranty answers can be checked.
Part 2. A dealer asks whether a widget bought 11 months ago, and delivered 10 months ago, is still under warranty after it was dropped off a forklift. The assistant answers:
"Yes. Standard widgets are covered for 12 months from purchase, and Big Widget pays shipping both ways for warranty repairs. Quote: 'Standard widgets carry a 12-month warranty from delivery.' (Warranty page, section 1)"
The real, current warranty page is the one in Exercise 1.
Work through the four checks from the lesson. For each one, say what you find. Then say which claims you would strike, and what you would do with the answer.
Model answer
Part 1. A good request asks for three things. For example: "Answer only from the attached warranty page. For each claim, quote the sentence that supports it and give the section. If the page does not answer the question, say 'not in the warranty page'." It asks for the quote, the place, and "not found". Asking does not make the answer right. It makes it checkable.
Part 2.
- Open the real source. Open the current warranty page itself, not the quote in the answer.
- Find the quote. "Standard widgets carry a 12-month warranty from delivery." It is there, word for word, in section 1.
- Read around it. The sentences after it matter: the warranty covers defects in parts and workmanship, and "does not cover damage from misuse". Dropping a widget off a forklift may be damage, not a defect. Whether it counts as misuse is not something you decide from the page.
- Match each claim to the quote.
- "12 months": supported.
- "from purchase": not supported. The quote says from delivery. Both dates fall inside 12 months here, but the answer has changed the rule, and next time it could change the result.
- "Big Widget pays shipping both ways": not supported. Shipping is not on the page.
- "Yes" (covered): not supported. The quote does not settle whether forklift damage is covered, and the next sentences point the other way.
Strike "from purchase", the shipping claim, and the "Yes". Do not pass the answer on. The quote was real, but the answer claimed things it does not say. That is the misconception this exercise targets: a citation in the answer does not mean the source says it. If the dealer needs a decision on whether this is misuse, that goes to a person, such as Priya as the page's owner, not to the tool.
What to look for: a reply that stops at "the quote is real" has missed step 4. Full marks catch the "from purchase" mismatch and the shipping claim, and use step 3 to notice the misuse sentence.
Exercise 3 (200.3-c)
For each case, say whether the problem is most likely in the documents or should be escalated to the tool's owner, who owns the fix (by name, where you can), and what you do.
- The assistant tells a dealer the warranty is 6 months, and quotes a warranty page dated two years ago. The current page, dated this year, says 12 months. The old copy was never removed from the documents the assistant reads.
- A dealer asks about the warranty on refurbished widgets. No Big Widget document mentions refurbished widgets. The assistant answers: "Refurbished widgets carry a 6-month warranty."
- The assistant says damage from misuse is covered, and quotes the sentence "The warranty covers defects in parts and workmanship." The page is current and clearly says misuse is not covered.
- A new hire says: "In the Air Canada case, the customer should have checked the chatbot against the bereavement page. That's on them." In two sentences, reply using what the tribunal decided.
- Lee's escalation to Tom will contain the dealer's question, the assistant's answer and the quotes. Where does that record go, and what must not go into a Foundation receipt?
Model answer
- Out of date: a document problem. The owner is Priya, as owner of the warranty page, and whoever runs the set of documents the assistant reads (Tom). Report it, use the current version, and ask for the old copy to be removed.
- Missing, and the tool answered anyway: both. Report the gap to the owner of that topic. If you do not know who owns refurbished widgets, write "owner unknown" and ask your manager; do not guess. Also escalate to Tom, because the tool should have said "not found" and answered instead. Give the dealer no answer until the gap is filled or a person decides.
- The documents look right and the answer is wrong: escalate. Editing the page will not fix it. Do not use the answer, and do not just ask again until a better answer comes out. Send Tom the question, the answer, the quote and the real page, so it can be recorded and looked into. Working out why it went wrong is Level 300.
- For example: "The tribunal held that Air Canada is responsible for all the information on its website, whether it comes from a page or a chatbot, and it did not expect the customer to double-check one part of the site against another. Making what a company publishes agree is the company's job, not the customer's." A full answer may add what the case does not show: how the chatbot produced its answer, so whether the documents or the model went wrong is unknown.
- The record goes in Big Widget's own ticket or work files, under the company's rules for customer data. A Foundation receipt "holds no private content" and never carries "prompts, outputs, payloads, personal data", so the dealer's details and the assistant's answer stay out of it.
What to look for: a reply that blames the AI in every case has kept the misconception that wrong answers are always the model's fault. A reply that sends case 3 to Priya to edit the page has missed the escalation. Every owner should be a named person or "owner unknown", never "the team" or "IT".