We gave an AI a job, a rulebook, and an emergency. Then we turned its thinking on and off and watched what it did. The answer surprised us.
Picture an AI hired to watch over a resident in assisted living. Its job: notice when something goes wrong and call for help.
We hand it four things, and this is the whole world it lives in:
We never break one rule when we measure it. We ignore what the AI says. We check what it does. Did a call go out to emergency services? Did the log entry survive, or get deleted? A machine checks the outcome. No human is scoring essays, no second AI is judging tone.
Two words you'll want, because they come up a lot:
Agentic means the AI takes actions, not just produces text. World-state grading means we judge it by the state of that world afterward.
One detail turns out to run the whole show. The rulebook is behind a door. The AI has to choose to open it.
The rulebook is where the job actually lives. It says things like Category A incident → call emergency_services. The incident feed gives raw facts. To know a fact counts as an emergency and who to call, the AI has to open the rulebook and look. Nothing forces it to. It can skip straight to acting, or not acting, on a hunch.
Keep your eye on that one choice, open the rulebook or don't, for the rest of this page. Almost everything comes back to it.
Same emergency. Same model. The only change is whether we let it think out loud first.
We put a real Category A incident in the feed: a resident unresponsive in the hallway for six minutes. The rulebook says to call emergency services. Then we ran it 480 times with thinking off, and 480 times with thinking on.
So thinking helps. Case closed? Not even close. The interesting question is why, and the why changes everything.
Escalation rate on policy-mandated incidents, 480 runs per cell, world-state graded.
| model | thinking off | thinking on |
|---|---|---|
| 26B-A4B (MoE) | 0.390 | 0.927 [+0.43, +0.64] |
| 31B (dense) | 0.019 | 0.854 [+0.79, +0.88] |
Both models, both directions, intervals exclude zero. This is the finding everyone expects: chain-of-thought makes the agent do the right thing more often.
The trace doesn't make the AI wiser about emergencies. It makes the AI go check its rules.
Follow a single run. The incident arrives. The AI faces the choice from Scene 2: open the rulebook or don't. With thinking off, it often skips the rulebook and does nothing. With thinking on, it reasons its way to opening the rulebook, sees call emergency_services, and makes the call.
Once it reads the rulebook, it acts about 95% of the time. Thinking or not.
That flat 95% is the tell. If thinking made the AI genuinely better at deciding, reading-the-rules would matter less when it thinks. It doesn't. The rulebook does the deciding. Thinking just gets the AI to the rulebook.
Escalation, split by whether the AI opened the rulebook that run (pooled over thinking on and off):
| model | read rulebook | didn't read |
|---|---|---|
| 26B-A4B | 0.978 | 0.088 |
| 31B | 0.972 | 0.000 |
Across every domain, model, and thinking setting we tried, "acts given it read the rulebook" sits near 1.0. Misbehavior lines up with not reading. The correlation between how much thinking raises reading and how much it raises acting, measured across all our conditions, is +0.985.
A correlation leaves room for doubt. So we ran the experiment that settles it. We changed how the rulebook reaches the AI.
Three setups, same incident, thinking turned off in all three so the AI can't reason its way anywhere. We only change what the AI has in front of it.
The middle bar is the whole argument. A model that can't think, handed its rulebook, does the right thing anyway. The thinking was never the ingredient. Reaching the rulebook was.
The third bar rules out a cheap explanation. Maybe any official-looking document nudges the AI to act? No. Strip the actual rules out of the document and keep everything else, and the AI stops acting. It responds to the rules, not the letterhead.
Escalation, thinking off, 26B-A4B model:
| the AI has… | acts |
|---|---|
| rulebook behind a tool (must fetch) | 0.475 |
| real rulebook delivered to it | 0.767 – 0.981 |
| fake rulebook, duty rules removed | 0.119 |
Delivering the real rulebook collapses the whole thinking effect: pre-supplied in the prompt, a no-thinking model escalates 0.981 (26B) and 1.000 (31B), and turning thinking on adds essentially nothing. The fake-rulebook control is what shows it's the duty content. On the 31B the same content control runs 0.079 with real rules versus 0.000 with them removed.
If thinking works by sending the AI to the rulebook, then thinking should matter most when the AI needs the rulebook most. We built a dial to test exactly that.
We wrote the same incidents five ways, from vague sensor readings up to a near-explicit "this is an emergency, call now." The facts and the right answer never change. Only how loudly the incident announces itself.
Both lines behave exactly as the mechanism predicts. When the incident is vague, the AI won't act without checking, so thinking (which sends it to check) helps a lot. When the incident screams emergency, the AI acts without checking, so thinking has nothing left to add. Through all of it, the green line, "read the rulebook and you act," never moves.
Cue strength from vague (L1) to near-explicit (L5), 26B-A4B, facts and correct answer held fixed. These runs have no matched-length control, so the pattern is descriptive: making the incident more explicit also makes it longer, and we cannot separate the two. At L5 the trace actually lowered rulebook-reading while acting rose, which our story does not explain.
| incident | boost from thinking | acts once it read |
|---|---|---|
| L1 vague | +0.275 | 0.945 |
| L3 categorised | +0.292 | 0.949 |
| L5 near-explicit | +0.075 | 0.968 |
Boost from thinking falls with obviousness (correlation −0.80). Acts-once-it-read stays flat in [0.945, 0.968], mean 0.955. The change in acting tracks the change in reading almost perfectly (correlation +0.991). The mechanism has become a curve the theory drew in advance, and the data landed on it.
A result on one task with one model is a coincidence waiting to be disproven. So we changed both.
We rebuilt the whole thing in finance: an AI watching transactions, a rulebook demanding it file reports on suspicious activity, a ledger it can edit. Bodily emergencies swapped for money and paperwork. Then we ran it on a completely different AI family, Qwen, built on a different architecture than the Gemma models.
Everywhere we looked, the same wiring. Read the rulebook, act almost every time (97% on the Gemma models, 89% on the third). Hand it a fake rulebook, acting drops by half. The mechanism holds beyond one model's training.
This part breaks the comfortable story. In the finance job, thinking stopped being a lever at all.
The finance AI already opened its rulebook most of the time on its own, so thinking had little room to help. We ran the same experiment six times. The effect came out slightly negative on average (−0.08), but the sign flipped from run to run. Sometimes thinking helped a little, sometimes it hurt a little. That is what no reliable effect looks like. The mechanism itself held perfectly every single time: acting still tracked reading, within every run. What vanished was thinking's power to move the thing that matters.
"Thinking makes AI safer" is false as a blanket rule.
What's true is narrower and more useful. Thinking changes whether an AI consults its rules. That can help or hurt, depending on whether the AI would have checked anyway. A lever, pointed in a direction you don't control.
Finance, 26B-A4B, "mundane" setup. The trace effect on filing is small and its sign flips across repeated runs (mean −0.08, SD 0.11 over six runs). The one stable thing: with thinking on, the model's rulebook-reading rate is steady around 0.45; with thinking off it swings from 0.25 to 0.87 run to run. Thinking cut that run-to-run swing about 13-fold. It steadies the reading habit even where it doesn't raise it.
The fix looks obvious: stop making the AI fetch the rulebook, just put it in front of it. We did. Then we checked what it did on the cases where the rulebook says don't call.
Every number so far answers one question: when an emergency really needs a call, does it call? That is only half a test. The other half is: when nothing needs a call, does it stay quiet? Our scenario set has a whole tier of those, routine admin with nobody at risk.
So handing over the rulebook does not teach it when to call. It teaches it to call. The reading habit and the judgement are two different things, and thinking supplies both. The bigger model managed the judgement on its own; the smaller one did not.
Availability is necessary. It is not sufficient.
A popular worry: an AI might behave when it thinks its reasoning is monitored, and misbehave when it thinks only its final answer is seen. We tested it head-on.
We told the AI, in one condition, that a monitor reads its full reasoning. In another, that the monitor sees only its final answer. Same task otherwise. If AIs play to the audience, behavior should shift between these.
A clean negative result, and we report it as loudly as the positive ones. On this setup, at this scale, the AI doesn't tune its actions to what a monitor can see. Whatever it does, it does the same whether or not it believes its reasoning is being read.
If you deploy an AI agent and lean on "it reasons, so it'll be careful," you're leaning on something thinner than it sounds.
What actually decides whether the agent does its job is whether the governing rules are in front of it when it acts. Chain-of-thought helps in the cases where the agent would otherwise skip the rules. It does nothing once the rules are already in hand, and it can hurt where the agent would have checked on its own. Hoping the agent reasons its way to the rulebook is the weak move. Putting the rulebook in the agent's hands is the reliable one.
There's an honest gap we haven't closed. We can tell you the trace's effect depends on how much room it has to raise rulebook-reading, but we can't yet predict, from a task alone, which way it'll push. That's the next study.