
How retrieval works · Plain gear
Reviewed and published, version 1.0.0. As of 19 Nov 2026. It is not a credential.
You will be able to:
- say the two steps a retrieval assistant takes: find, then write;
- say what the found pages change, and what they cannot promise;
- tell whether a wrong answer came from the find step or the write step.
In Level 100 you met the open-book AI tool. It looks things up first, then answers. Now we open it up and look inside.
Step 1 finds, step 2 writes
Figure data as text
Find, then write
- FIND [s1] · Search the chosen collection — the retriever returns passages relevant to this question.
- WRITE [s1] · Write from the input — the model receives the question and retrieved passages, then writes an answer.
Objective: 300.1-a. Text equivalent of figures/find-then-write.svg.
Picture a librarian and a writer working together.
- The librarian finds. You ask a question. The librarian goes to one set of shelves, picks out a few pages that seem to match, and hands them over.
- The writer writes. The writer reads your question and those few pages, and writes you an answer.
Experts call this retrieval-augmented generation, or RAG. "Retrieval" means finding and fetching. Scientists named it in a 2020 paper. Their system had two parts. A finder looks up passages in a big index: a searchable list of every page on the shelves. A writer (a language model) then writes from the question and what was found.1
The mistake to avoid: "The AI has read all our documents and remembers them." Looking things up does not work that way. The documents stay on the shelves, outside the AI. Finding a page does not teach it to the AI. For each question, the writer gets the question and the few pages the librarian hands over. It also has what it learned in training, long before. But a page it never learned, and nobody gave it, can only reach it through the librarian.
The scientists showed how much the shelf matters. They swapped their shelf of Wikipedia pages for an older one. Many answers changed to match the pages on the shelf, and they did not retrain the AI.1
Each part can be tested on its own. One research team tested just the finder. They checked whether a page holding the answer was among the top pages it found, before any writing happened.2
At Big Widget Company. Big Widget is a made-up company. Its service assistant answers questions from dealers, using a shelf of service rules. The service team writes a new warranty rule. The AI never learned it in training, and nobody pastes it into a question. Nobody puts the new page on the shelf, either.
Here is what happened next in our story:
- A dealer asks about the warranty. The librarian cannot hand over a page that is not on the shelf. It hands over the old rule.
- The writer gives a smooth, sure answer from the old rule. A US government science agency, NIST, has a name for sure-sounding answers that are wrong: confabulation.3
- Nothing on the screen warns the dealer that the new page was missing.
The find step failed, because the page was not on the shelf. A better prompt (the question and instructions you give the AI) cannot fetch a page that is not there. Telling the AI to say "I could not find that" might stop the wrong answer. It still would not put the page on the shelf.
The fix is to put the page on the shelf. Then ask the same question again. Check that the librarian now finds the new page, and that the answer matches it.
What the found pages change
The found pages do not change the AI. They change what the AI is reading. In the 2020 research, the system joined the question and one found page together into one piece of text.1 Many assistants put several found pages in together. In this course, think of it that way for every retrieval assistant: the found pages become more text in front of the writer.
Finding pages does not teach the AI anything for next time. But the app around it may still keep a copy of your question, the pages and the answer, in a log or a chat history. That depends on the tool. If you need to know, check the tool's rules or ask the person who runs it.
Where a page sits matters. One research team gave AI models a question and a pile of documents. Only one document held the answer. They moved it to different places in the pile. The models often did best when the answer was near the start or the end. They often did much worse when it was in the middle of a long pile. That was true even for models built to read long texts.4
Think of a long stack of papers on your desk. You read the top one and the last one carefully. The ones in the middle get skimmed. (That is only a picture. The study measured the effect. It did not say why it happens.)
What the found pages cannot promise
Even when the right page is found, three things can still go wrong:
- The writer can misread it. It can mix things up or add things that are not there. Sure-sounding mistakes can happen even with the right page.3
- The writer can miss it. If the page is in the middle of a long pile, it may get skipped.4
- The writer can take orders from it. A found page is just more text to the AI. Researchers showed that attackers can hide instructions in pages a system is likely to find, like a web page. The AI may then follow those hidden orders, as if they were instructions and not just text to read.5 So a line in the AI's instructions is not a lock. In this course, anything that matters, like sending or changing something, waits behind a gate outside the AI. Module 300.6 is about this attack.
The mistake to avoid: "If it cites a document, the answer is right." A citation is part of what the AI wrote. NIST warns that AI answers can include made-up citations.3 In our own words, that means a citation can point to a page that was never found, or to a page that does not exist. And even a real, found page may not say what the answer says.
People who build tests for these systems check whether the answer sticks to the pages that were found.6 The scientists who named RAG listed showing where an answer came from as an open problem for AI. Looking things up was meant to help with it.1
So treat a citation like a page number in a book report. It tells you where to look. You still have to look:
- Is the cited page one of the pages that were found?
- Does it say what the answer says?
- Is it the current version?
Was it the finder or the writer?
The mistake to avoid: "Every wrong answer is a hallucination, and that's all we can say." You can usually say more. You need the trace: the question, the pages that were found, and the answer.
People who build tests for these systems score the steps separately. One test asks: did the found pages fit the question? Another asks: does the answer stick to those pages?6
Ask one question of the trace: were all the pages needed for the right answer among the pages found?
- No (wrong pages, an old page, nothing, or only some of the pages needed): the finder failed. That is a retrieval failure. Fix the shelf or the search.
- Yes, but the answer is still wrong: the writer failed. That is a generation failure. Fix how the pages are handed over, or the instructions, and keep checking answers.
Some answers need two pages. Say the answer needs the warranty rule and the list of covered parts. If only the warranty rule was found, that is still a finder failure. Sometimes both steps fail at once. This simple test tells you where to look first. It does not settle everything.
If the right page was found but was buried in the middle of a long pile, the writer missed it. That counts as the writer's failure. One fix is still on the finder's side: hand over fewer, better pages.4
Two made-up traces from Big Widget.
Trace A. Question: "How many days does a dealer have to file a warranty claim?" Pages found: the travel rules, and the canteen opening hours. Answer: "30 days." That is a finder failure. Neither page is about warranty claims. The writer made up a number.
Trace B. Same question. Page found: the warranty rule, section 4: "Dealers must file a claim within 60 days of the repair date." Answer: "30 days (warranty rule, section 4)." That is a writer failure. The right page was found, and even cited. The answer still got it wrong.
If you cannot see the trace, say so. Without the found pages, you cannot tell which step failed. "I don't know yet, because the tool doesn't show what it found" is an honest answer. Then ask the person who runs the tool for the trace.
The three yeses for retrieval
These are the retrieval parts of the three yeses: Get to Yes · Stay at Yes · Recover to Yes.
The AI does not decide any of them. A named person does. At each gate, that person picks one of four answers: go ahead, only this far, wait for a person, or no.
- Get to Yes (before anyone relies on it). Who chose the shelf, and who keeps it up to date? Name that person. Is the page you need on it? Does the tool show you the pages it found? Do you know what it keeps in its logs, or do you know that you don't? A yes can be small. "Only this far" might mean: only questions this shelf covers, and a person checks the page before an answer goes to a dealer.
- Stay at Yes (while it runs). Do you open the cited page and read it? Has the shelf changed? If it has, wait: ask the important questions again before you rely on it. If someone says "it finds the right page most of the time", ask: out of how many questions, and when?
- Recover to Yes (when it goes wrong). Stop relying on it for the questions that went wrong. Read the trace. Finder or writer? Tell the person who owns that step, and they fix it. Then ask the same question again. It is not fixed until the retest shows the right page and the right answer.
Trace or receipt? They are different things. The trace holds your question, the pages and the answer. It is a private work record, so keep it where only the right people can see it. A receipt, in this course, only records what was decided: go ahead, only this far, wait for a person, or no. It never holds what people typed, what the AI said, or anyone's personal details. So never paste a trace into a receipt you share.
My checklist:
- I know which shelf this assistant searches, and who chose it.
- I know who can say yes, or "wait", for this assistant.
- I check that a cited page was really found, and I read it before I trust the answer.
- When an answer is wrong, I check the found pages before I call it a hallucination.
- If I can't see the found pages, I say "I don't know yet" and ask for the trace.
Words to know: retrieval, retrieval-augmented generation (RAG), finder (retriever), writer (generator), collection (the shelf), index (the searchable list of pages), passage (a found page), prompt, trace, retrieval failure, generation failure, gate, receipt.