
Reading and critiquing primary research · Plain gear
You will be able to: check whether a research paper's big claim is backed by its own evidence; tell a reviewed paper from a draft, a company report and a safety card, and say what each one can and cannot prove; and spot three ways a test score can fool you.
This module is as of November 2026. Reviewed and published, version 1.0.0. It is not a credential.
1. The headline is a claim, not a fact
Think of a film trailer. It shows the best bits. It is made by the people who want you to buy a ticket. A research paper's summary, called the abstract, is a bit like that. The authors wrote it. They chose what to put in it. To know what they really showed, you have to look inside.
Two researchers, Zachary Lipton and Jacob Steinhardt, warned about this in 2018. They said papers often dress up guesses as explanations. Readers then believe the guesses, because they appear in a scientific paper written by experts.1
Four questions to ask
- Better than what? A claim that something is "better" needs a fair comparison. Was the old way given the same care as the new one?
- Tested on what? Which examples? How many? Are they like the real situation?
- Scored how? What number was used? Out of how many? Was the test run once, or many times?
- What would "wrong" look like? Before you look at the results, picture what they would show if the claim were false. If no result could ever prove it wrong, it is not really being tested.
The fourth question is this course's own rule. It is the one that turns reading into checking.
Real case: the unfair race
Imagine a race where one runner trains for months and the other does not train at all. The trained runner wins. Does that prove their new shoes are better? No.
Something like this happened in AI research. In 2017, three researchers re-tested several newer designs for language models against an older design called an LSTM. They said the earlier tests had used different code and limited computer power. This time, every design was tuned in the same careful, automatic way. The older design, set up properly, beat the newer ones on the tests they used.2 So some earlier "improvements" had come from better tuning, not from the new designs.1
That is true for those models and those tests. It does not mean tuning explains every claimed gain. (We read only the summary page of their paper, posted on a site called arXiv. That page lists no other version.2 But Lipton and Steinhardt's list of sources names a version at a conference, ICLR, in 2018.1 We have not opened that version, so whether it was reviewed is not checked. What we quote here comes from the arXiv page.)
Real case: predicting wars
Some researchers had published papers saying clever computer models could predict civil wars much better than a simple, old method. In 2022, two researchers at Princeton, Sayash Kapoor and Arvind Narayanan, checked those papers again.3
They found mistakes in how the tests were set up. In some, information from the test had leaked into the training. In one, the model was given clues that were close to the answer itself.3 When the mistakes were fixed, the clever models did not do much better than the simple, old method.3
These papers had appeared in top journals. And the researchers said none of the mistakes could have been caught just by reading the papers.3 They had to dig into the data and code.
Real case: same test, run two ways
A team that builds a popular tool for testing AI language models tried something simple. They gave one model the same multiple-choice questions, run in two different ways. In one, each possible answer was scored directly. In the other, the choices were listed as A, B, C and D, and only the letter the model picked was scored. With the first way, it got about 72 out of every 100 right. With the second, it got about 26.4 Same model. Same questions. Only the way the test was run changed. They warn that even tiny changes, down to spaces in the question, can change the score.4
The mistake to avoid: "If it is in a paper, it is a fact. The summary tells me what they found." A result is what happened in one test, set up one way. Check the four questions yourself.
At Big Widget Company
Made-up example. A supplier says its camera system "beats human inspectors at finding bad welds". The quality team looks inside the report. The system was tested on 500 photos from the supplier's own factory. The "human" score came from two people who each looked at only 50 photos.
- Better than what? Not a fair race: different people, different photos.
- Tested on what? Photos from another factory, not ours.
- Scored how? If 95 out of 100 welds are fine, a system that says "fine" every time scores 95 out of 100. How many bad welds did it miss, out of how many bad welds?
- What would "wrong" look like? Before the test, the quality lead decides what "better" must mean. For example: it catches more bad welds, without more false alarms, by enough to matter. Then test the system and the people on the same photos. This report cannot show either way.
Watch out for a near tie. If a fair test comes out almost even, that does not prove the claim wrong. It means the test has not shown the system is better. To count against the claim, the test has to be sure enough to rule out a gain that matters. Even then it is strong evidence, not a promise. A close test that is not that sure gives an honest answer: "we don't know yet".
The team writes: "Supplier claim. Not checked on our welds. Next check: system and inspectors on the same photos." The quality lead owns that next check. Until it is done, the claim waits. It is not used in any decision.
2. What the label on a source can and cannot prove
Sources come with labels. Each label tells you how the document was made. None of them tells you the claim is true.
A reviewed paper
Other experts read it before a journal or conference accepted it. That is useful. But it is not proof.
- In 2014, one big AI conference tried an experiment. About one in ten papers was sent to two separate groups of reviewers. Across all those papers, the groups disagreed on about one in four. Among the papers they accepted, they disagreed about more than half (57 out of every 100).5
- Reviewers are not expected to re-run the experiment. At that conference in 2019, code was only expected for accepted papers, after review. It was not expected to be used to judge whether the work was sound.5 That was the rule. Some reviewers did look at code that was shared, and many found it useful.5
- The war-prediction papers above were reviewed and published. They still failed when re-checked.3
A draft (preprint)
A preprint is a paper the authors posted online themselves, often on a site called arXiv. It shows what the authors claimed, and when. In this course we treat it as not reviewed, unless you find a reviewed version.
Papers also come in versions, like drafts of an essay. Lipton and Steinhardt's 2018 draft was written for a workshop debate.1 A piece with the same title came out in ACM's Queue in 2019.6 We have only seen its listing, not the piece itself. So we do not know if it was reviewed: not checked. They are different documents. Always note which version you read.
A company report
A company writes about its own AI. That tells you what the company chose to share. One AI company's 2023 report on its own model said it left out details such as the model's size and how it was trained.7 So that report alone does not give outsiders enough to rebuild the model the same way. People outside may still be able to test some of its answers, if they can use the same version of the model.4 A test like that checks those answers. It does not check how the model was built. One team of researchers put the bigger problem simply: AI makers often "grade their own homework".4
A safety card (system card)
A system card is the company's own account of the risks it found and the safety steps it took. The same company's system card says its safety measures "are limited and remain brittle in some cases".7 A system card tells you what the company says it did. It is not an outside check.
The mistake to avoid: "It was reviewed, or it has an arXiv number, so it must be true." Reviewed means some experts accepted it. An arXiv number means it was posted. Neither one re-ran the test.
At Big Widget Company
The team keeps a list of every source they rely on. For each one they write: what kind it is, which version they read, who wrote it, and what they could not check. The supplier's weld report looks like a science paper. It goes on the list as "company report, not reviewed, not tested by anyone else".
3. Three ways a test score can fool you
A benchmark is a fixed test that lots of researchers use to compare AI models.8 A high score can come from the wrong reasons. Look for three problems.
1. Leakage: the test was not really kept apart
Imagine you want to know if a student can name birds from photos. You give them a photo of a robin to practise on. Then, in the test, you show a second photo of the same robin, taken a second later. The student gets it right. Did they learn robins, or just remember that one bird? You cannot tell.
In AI, this is called leakage. Information from the test gets into the training, or the model is given a clue that would not exist in real life. Kapoor and Narayanan found leakage in every one of the reviews of problem papers they collected, from 17 different fields of science.3 One example: a model meant to predict high blood pressure was given, as a clue, whether the patient took blood pressure pills.3
2. Contamination: the model may have already seen the test
Contamination is one kind of leakage. Big AI language models learn from huge amounts of text taken from the internet. Test questions are often online too. So the model may have read the test, and even the answers, before it is tested.9 How big the problem is "is unknown", researchers say, because it is hard to measure.9
A test being online is a risk, not proof that the model read it. Keep those apart.
The company's 2023 report describes how it checked its own exams. It used a simple matching tool to find questions the model might have seen. It ran each exam again without those questions and reported whichever score was lower.7 It also found that parts of one test set had slipped into its training data by mistake, so it left that test out.7 But the company did the checking itself. It says its matching tool can miss questions the model did see, and can flag questions it did not see.7 So even the lower score is not proven clean.
3. The test does not match the job
A driving test in an empty car park does not show you can drive in city traffic. Researchers call this a construct validity problem: the test does not measure the real thing you care about.8
Inioluwa Deborah Raji and colleagues compare some famous AI tests to a children's story. In it, Grover visits a museum that claims to hold "everything in the whole wide world". It has a few odd rooms. When he opens the last door, marked "Everything Else", he is outside.8 No fixed test can hold everything. In 2021, a big tech company said its model beating humans on one language test was "an important milestone toward general AI".8 That is a huge claim from one test.
The mistake to avoid: "A higher test score means the AI is better at the real job." A higher score means it did better on that test. It tells you about the real job only if the test was kept apart from the model and looks like the real job.
At Big Widget Company
The team looks again at the weld report.
- Leakage: each weld was photographed five times. Some photos of the same weld were used for practice and some for the test. That is the robin problem.
- Contamination: the report does not say whether the system had seen a public weld-photo collection it was also tested on. That is a risk, not a finding. Not checked. The quality lead asks the supplier.
- Does the test match the job? The test photos are bright and clear. Big Widget's factory cameras are not.
The team asks the supplier to test the system on our own camera photos of welds it has never seen. Until then, the claim stays "not checked".
4. Make it a habit
The Foundation has three yeses: Get to Yes, Stay at Yes, Recover to Yes.10 They are a cycle, not a one-time okay. At a Foundation gate, the gate decides, and there are four answers: "go ahead · only this far · wait for a person · no".10 You, the reader, do not decide. Your job is to say what the evidence shows, and what it does not.
- Get to Yes: before a research result shapes a decision, write down what kind of source it is and which version. Ask the four questions and look for the three problems. Write "not checked" next to anything you could not confirm. Name the person who decides. If something important is not checked, the result waits for that person. It does not go in by default.
- Stay at Yes: papers get updated. One of the papers in this module was on its third version in 2026.4 Check again when a new version comes out.
- Recover to Yes: if a result you relied on turns out wrong, stop using it, say so, name who will fix it, and look at the decision again. A plan to re-check is not the same as having re-checked.
Even checking the original computer code is not a guarantee. Researchers who ran a big checking programme point out that the code can have mistakes too.5
The Foundation says "I don't know yet" is always an answer, and it always pairs you with someone.11 So when you write "not checked", also write who will check it next.
What you keep. Keep your notes (what you read, which version, what you could not check, who decided) in your own work record, where your workplace controls who can see them. The Foundation's receipt is different. It is the record of a gate's decision, and it never holds "prompts, outputs, payloads, personal data".11 So what you typed, what the AI wrote, and anyone's private details stay out of it.
Words to know: abstract, claim, fair comparison, reviewed paper, preprint, version, company report, system card, benchmark, leakage, contamination, construct validity, not checked, gate, receipt.