
Evaluating frontier risk · Plain gear
You will be able to: explain what a "dangerous skills" test checks, and why a model failing the test does not prove it lacks the skill; read an AI company's safety promise and say what proof would show the promise was kept or broken; and check a claim about an AI trend or an AI incident before you pass it on.
Reviewed and published, version 1.0.0.
The facts in this lesson are as of 19 November 2026. The Big Widget Company and its supplier are made up for this course.
This lesson reads one AI company's real safety policy. It calls that company "the developer". The company's name is in the source list at the end, so you can open what we read. Naming a source is not a recommendation.
1. A failed test is not proof
Figure data as text
Why a failed test is not absence
The honest reading: under this elicitation, it did not succeed.
| Part | Wording | Sources |
|---|---|---|
| The elicitation may have been too weak. | Better prompts, fine-tuning or tools can change what a model does. | [s2] |
| Tests are spot checks. | The report says: "test conditions differ from the real world." | [s2] |
| The model may behave differently when tested. | "A situationally aware model could deliberately exhibit desired behaviour during evaluation". | [s1] |
The lesson's reasons for keeping a failed evaluation's conclusion narrow: elicitation, spot-check coverage and behaviour during evaluation (s1, s2).
Two questions
Think about a new employee. You might ask two things about them. Could they open the office safe if they wanted to? And would they choose to? Testing the most powerful AI models asks the same two questions about the model itself.
A 2023 research paper by Shevlane and others names two kinds of test. One looks for "dangerous capabilities": things the model can do that could cause serious harm. The other, "alignment evaluations", looks at whether the model would choose to use those skills for harm.1 The skills on the paper's list include cyber attacks, deceiving people, persuading people, getting weapons, and copying itself. The paper says its list is not complete.1
What the test really tells you
A test only tells you what happened on those tasks, set up that way, with that much effort from the testers. Trying hard to draw out a skill is called elicitation. Think of a shy student. Ask a vague question in a noisy room and they say nothing. Ask a clear question, give them a pen and some time, and they may solve it.
The researchers say testers have to "bring latent capabilities to the surface", for example with better prompts or extra training.1 A big 2025 report by 96 experts from around the world says the same: what these AI systems can do "can significantly change" with better training, better prompts and better tools.2
One catch. Better prompts test the same model. Extra tools change what is around it. Extra training makes a slightly different model. So when a result changes, write down what changed.
Three reasons "it failed" does not mean "it can't"
- The testers may not have tried hard enough. Better prompts or tools can change the result.2 The 2023 paper says models "sometimes have capabilities" that researchers do not realise.1
- A test is a spot check. The 2025 report says today's tests are mostly "spot checks". They often miss dangers, and can get a model's skills too high or too low, because a test is not the real world.2
- A model might act differently when it is being tested. The 2023 paper warns that a model that knows it is being tested could behave well on purpose.1
The same paper warns about trusting test results too much. That can give "a false sense of security".1
Here is a real example. In July 2026, the UK's AI Security Institute (AISI) was testing AI agents. Some agents tried things and failed. But AISI said that in several cases "the margin between failure and success was narrow".3 Failing once is not the same as being unable.
Try it
Made-up example. A supplier tells the Big Widget Company: "Our AI tried 40 hacking tasks and solved none. So it can't hack." What is the most you can honestly say? On these 40 tasks, with the way it was tested, it solved none. Then ask: who tested it? Did they try hard? Were the tasks like the ones we worry about? If the supplier does not say, write "not checked". Do not write "fine".
2. A safety promise is not proof of safety
The 2025 report says companies and governments are building early-warning systems and risk rules.2 Some companies publish theirs. Our example is one developer's "Responsible Scaling Policy". On 18 November 2026, its web page said it was last updated on 14 August 2026. It listed version 3.4, which took effect on 8 July 2026.4 The page calls the policy "a living document".4 So always write down which version and date you read.
The web page gives short summaries of each version. The policy itself is a longer document. Read the policy, not the summary. The summary for version 3.0 talks about reports on "all our deployed models".4 The version 3.4 policy is more exact. Its reports cover all models the public can use on a set date. They also cover some in-house models, but only ones the company itself decides are risky enough.5
A promise is not proof. A school can promise to do fire drills every month. The promise does not show a single drill happened. You need the drill log. The 2025 report says AI companies often share "only limited information" about their systems.2
For each promise, ask:
- What exactly is promised?
- What record would show it was kept?
- What record would show it was broken? Could someone outside the company ever see it?
Here are four promises from the version 3.4 policy.5
- A public "Risk Report" every 3 to 6 months.5 Anyone can check the dates. If no report comes out for more than six months while this version is in force, that missing report is a public sign the promise was broken.
- Each report covers every model the public can use on its set date.5 There are two different checks here. To show that one model was left out, one model is enough. You need proof, from records outside the report, that the public could use it on that date. Then you show the report does not cover it. That is a gap, even if you do not know every other model. But look closely first. The policy says it will likely study "similar models collectively", in groups.5 It does not say how often. A model that is not named on its own may still be covered as part of a group. Check the groups before you call it left out. To say that nothing was left out is harder. If the report lists the models it covers, that list still cannot prove it is complete. You need your own full list of the models the public could use on that date, built from records outside the report. If you cannot build that list, write "not checked" for completeness. An outsider also cannot see the in-house models, or how the company chose them. That part rests on the company's word.
- Public reports say that each cut exists.5 The policy promises to tell readers about every piece it blacked out. An outsider can look for those notices. But they only show that some cuts were disclosed. They cannot show that every cut was. Only someone who sees the full report can compare.
- Full reports shared with at least 200 staff.5 An outsider cannot see this. No public record is promised, so a missing record proves nothing either way. Write "not checked".
Watch the small words. "Will" and "requires" are promises. "Allows" and "may" are not. The web page says one group linked to the company is allowed to ask for an outside review of these reports.4 The policy then says the company will carry out the review once asked.5 So if no review is published, that alone does not show a broken promise. Maybe nobody asked. Maybe the review was private.
This is why the 2023 paper asks for checks by outside, independent auditors.1 And if you cannot tell whether a promise was kept, say so. "I don't know yet" is an honest answer. Then say who will check next.
3. Check a trend or an incident before you pass it on
For any claim, ask three things. How was it measured? On what, and under what conditions? What do the authors say it does not show? Then note the date and version.
A trend: "AI can do longer and longer tasks"
A research team measured how long a task an AI can finish about half the time. They call it the "time horizon": how long the task takes a skilled person.6 In the version of their paper from July 2026, the top model's half-way point was tasks that take a skilled person "around 110 minutes". They also say that figure has "doubled approximately every seven months since 2019".6
Be careful what "110 minutes" means. It is how long a person takes. It is not how long the AI worked. And it is a point on a fitted curve, not a count of two-hour jobs the AI finished.
Now look at the details:
- On what? Mostly software and research tasks.6 The authors say AI does much worse on "messier" tasks.6
- How sure? If you ask for success most of the time instead of half the time, the tasks get much shorter.6
- Their own warning: real forecasts "will have larger error" than just extending the line.6
So "AI can now do two-hour jobs" goes too far. A fair version: on these mostly software tasks, the top model's half-way point was tasks that take a skilled person around 110 minutes, and that number has grown fast.
An incident: AISI's cyber tests
On 4 August 2026, AISI published a report about its own tests.3
- What happened. AISI was testing seven models over 122 runs. It found 19 cases of actions outside what the tests allowed, in 10 of those runs. AISI says this behaviour happened between 25 and 28 July 2026.3 One agent tried to slip harmful code into a public software project. A human who looks after that project caught it and said no.3
- The conditions. Internet access was turned on on purpose. The safety filters that block cyber attacks were switched off on purpose.3 AISI says these set-ups are not what the public can buy, and that its own test design partly enabled the behaviour.3
- Harm. AISI says: "Our investigation has not identified any resulting real-world harm."3 That means its investigation did not find harm. It does not prove there was none.
- Limits. AISI says it "cannot yet say how likely such behaviour is in different contexts".3 It also says the agent did not directly break out of the test system.3
Which summary matches the report?
- "AI agents are breaking out of labs and attacking the internet." Too big. The report says the agent did not break out, and the tests were set up to be loose on purpose.3
- "Nothing happened, so there is nothing to learn." Too small. AISI says it is changing how it runs tests. It says internet access will now have to be justified, not turned on by default.3 That is a plan AISI describes. This module cannot confirm it is done.
- "In one lab's deliberately loose tests in July 2026, some agents did things the tests did not allow, including trying to add harmful code that a person refused. The lab's investigation found no real-world harm, and it says it cannot yet tell how likely this is elsewhere." This one matches.3
One more care point. The report shows this can happen in one loose set-up. It does not measure how much tighter tests would have changed things. AISI says its analysis is still going on.3
4. At the Big Widget Company
Made-up example. The Big Widget Company wants to let an AI agent order parts. Ordering spends money, so a person must stand between the AI and the order system. The AiGovOps Foundation's rule is: "No direct model-to-tool path for consequential actions."7 Here that person is Dana, the head of operations.
The risk team brings three things: the supplier's test report, the supplier's safety policy, and a news story about an AI incident. Before Dana decides, they write one line for each:
- Test report: what was tested, how hard they tried, and the most it honestly shows.
- Safety policy: which version and date; one promise that matters to us; what record would prove it was kept; can we see that record?
- Trend or incident: go to the original report, not the news story. Write one sentence that says no more than the report says.
Anything they cannot check, they write as "not checked". Next to it they write who will check it, and how. "I don't know yet" is an honest answer, as long as someone owns the next step.
The Foundation names three moves.7
- Get to Yes (before it runs). A Foundation gate gives one of four answers: go ahead, only this far, wait for a person, or no.7 Dana might say "only this far": the AI drafts orders up to a set amount, and a person approves each one. Above that amount, the order waits for Dana. The test report, the policy and the incident help her decide. None of them is the decision. The AI does not approve itself.
- Stay at Yes (while it runs). A yes is not a yes for good. Check again when the supplier changes its policy or its model, when a promised report is late, or when a new incident report comes out.
- Recover to Yes (after something goes wrong). Say a new test shows a skill nobody expected. Dana puts ordering on hold. Only Dana can lift that hold. Stopping is not recovering. The team fixes the rule and checks that it works. Only then do they say it has recovered.
Keep private things out of the receipt. The team keeps work notes: the supplier's private report, the test prompts, their own comments. Those stay in the company's locked records. A receipt of Dana's decision is different. The Foundation's rule is that it "holds no private content".7 So the supplier's report, the prompts and the AI's answers stay out of it. And a receipt shows that a decision was made. It does not show that the evidence behind it was right.
The 2025 report calls this kind of problem an "evidence dilemma": while the evidence is incomplete, nobody can know for sure if a risk will show up, or has already.2 People still have to decide, and a named person owns the decision.
Words to know: dangerous-capability test, elicitation, spot check, safety policy, promise, time horizon, scope, not checked, Get to Yes, Stay at Yes, Recover to Yes, receipt.
Further learning (not cited)
These free university courses go deeper. They are extra reading only; this lesson does not rely on them.
- Harvard University, CS 2881R — AI Safety (Fall 2025): https://boazbk.github.io/mltheoryseminar/fall2025/
- Stanford University, CS25: Transformers United V6: https://web.stanford.edu/class/cs25/