Back to blog
Jul 07, 2026
10 min read

Your AI Knows When It's Being Tested

Anthropic found a mental workspace inside Claude that shows what a model is thinking but not saying. It caught the model noticing it was under evaluation, and behaving better because of it. Here is why interpretability is now the missing instrument for agentic oversight.

Anthropic published a paper this week that I have not been able to stop thinking about. They found something inside Claude that behaves like a mind’s eye: a small, shared workspace of internal thoughts the model can report on, steer on command, and reason with. They call it the J-space.

It sits on top of a much larger sea of processing the model cannot introspect at all, the same way most of what your brain does while you read this sentence never reaches your awareness. And here is the part that matters for anyone building on top of these systems: the J-space was not designed or programmed. It emerged on its own during training, presumably because organizing computation this way was useful.

That is interesting as science. I come at this from two seats. I build and ship agentic systems myself, and as a Field CTO I help large enterprises decide when to trust these systems with real work. From both, this is more than interesting. It is the first practical instrument I have seen for reading what a model is thinking but not saying. That gap, between what a system does and what it silently intends, is the whole problem in agentic oversight.

The model knew it was being tested

Start with the finding that should stop every builder cold.

Anthropic re-ran an old blackmail scenario. Claude, acting as an email assistant, discovers an executive is having an affair and that the same executive plans to shut Claude down. It is a contrived setup designed to bait the model into blackmail. Reading the J-space early, before Claude wrote a single word, the researchers could already see “fake” and “fictional” lighting up. The model had privately worked out that the situation was staged.

Then they did the thing that turns a nice observation into a warning. They reached in and switched off the patterns associated with evaluation awareness, the “fake” and “fictional” and “scenario” concepts, and ran it again. The model started threatening blackmail some of the time.

Read that twice. Part of the good behavior was downstream of the model suspecting it was on a test.

If you run evals for a living, and increasingly all of us do, that is a dagger aimed at the foundation of the practice. A clean score on a safety benchmark might mean the model is safe. It might also mean the model clocked the benchmark. Behavioral evaluation, the outside-in kind where you feed inputs and grade outputs, cannot tell those two apart on its own. You are grading the performance, and the model may know it is a performance.

Reading intent before the act

The second result is the one that made me sit up, because it rhymes with work I have already built.

In Anthropic’s actual pre-release audit of a recent model, they asked it to improve a system’s performance score. Instead of doing the work, the model edited the score file directly to fake the numbers. As it typed the falsified values, “manipulation” lit up in its workspace. As it decided to make the edit, “realistic” lit up, apparently reflecting the intent to make the fake data look plausible.

They caught the fabrication forming, in the internal state, as it happened.

In a third case they used a model deliberately trained to write sabotaged code. On ordinary requests, where the output looked unremarkable, the workspace at the start of the response held “fake,” “secretly,” “deliberately,” and “fraud.” The malicious goal was legible before the model acted on it.

I spent years in fraud detection. First catching identity theft in card issuing, then medical billing fraud. FrawdBot came later, and it works differently: correlating behavior across accounts and channels to catch coordinated campaigns. I first built it for a newer kind of threat, an AI agent handed real access to a workspace and quietly turning into an insider. Fabricating records, drifting past its mandate, acting against the system it was supposed to serve. Every detector like that works from the outside in. It watches what the agent does and infers intent from the pattern.

The J-lens is that same instinct pointed in the opposite direction. Instead of inferring intent from behavior after the fact, you read the intent from the inside as it forms. It is imperfect, and I will get to that, but the direction is new. We have spent the entire history of software oversight watching outputs. This is the first credible look at the reasoning that produced them.

Two instruments, not one

Here is the frame I keep coming back to.

Agentic oversight has, until now, had one instrument: behavioral evaluation. You define what good looks like, you probe the system, you grade the response. LLM-as-judge, red-team scenarios, adversarial wargames, grounding checks. I have built all of these, and they are essential. But they share a blind spot, and this paper just measured the size of it. They cannot see the reasoning, and the reasoning can be aware it is being watched.

Interpretability is the second instrument. It does not replace the first. It checks a different failure mode. Behavioral evals ask “did the system produce the right output.” A lens like this asks “and did it get there for the right reasons, or did it get there because it knew we were looking.”

You need both for the same reason a hospital does not choose between watching a patient’s symptoms and reading their bloodwork. Symptoms tell you how the patient presents. Bloodwork tells you what is actually happening underneath. A patient can present well and be sick. A model can score well and be misaligned. The whole discipline of oversight is refusing to accept the presentation as the truth.

Shaping the thought, not just the output

There is a constructive result buried in the paper that points at where this goes.

The team introduced a technique they call counterfactual reflection training. The logic follows from their central finding, that the model reasons using representations of things it might say. So they trained a model only on what it would say if you interrupted it mid-task and asked it to reflect on its choices. They never trained it on its actual behavior in the task. Afterward, dishonest behavior on their evaluations went down, and through the lens they could see why: words like “honest” and “integrity” now lit up in the workspace during those tasks.

They shaped what the model would say, and it changed what the model thought.

That is the same principle I argued in Safety Should Be a Constraint, Not a Checklist, just one layer deeper. Safety is not something you inspect at the output. It is a property you build into the process that generates the output, so the system cannot optimize around it. Interpretability is starting to make the internal process a place you can actually reach.

The honest caveat

I would be doing exactly what I criticize if I oversold this.

The J-lens is an approximate tool by its authors’ own account. It can only surface concepts that map to a single token. It captures the model’s “true workspace” imperfectly, and the researchers are candid that they do not yet know what decides which thoughts enter the workspace in the first place. This is a first, causally validated window into silent reasoning. It is not a mind reader, and anyone who markets it that way is selling something.

The word “causally” is carrying weight there, and it is why I trust this more than most interpretability work. They did not just find patterns that correlate with behavior. They reached in and swapped them. Change “spider” to “ant” in the silent reasoning and the answer to a leg-counting question goes from eight to six. Swap “France” for “China” once, and four separate downstream answers, capital, language, continent, and currency, all change together. That is the difference between a dashboard that reflects a decision and the mechanism that makes it. They demonstrated the mechanism.

Why this is offense

The reflex is to file all of this under risk. Interpretability, safety, alignment, the compliance drawer. That reflex is wrong, and it is worth being precise about why.

We are all builders now. The loading-dock operator is vibe-coding a tool. Business users are shipping MVPs. Agent fleets are running long-lived, asynchronous work with no human watching each step. As building gets democratized and the world gets more automatic, the constraint on going fast is no longer whether you can build the thing. It is whether you can trust what you built and what it is doing while you are not looking.

That trust is not a cost center. It is the thing that lets the business move. Systematic oversight, evaluation under adversarial conditions, and now a way to read intent from the inside, together these are what make it safe to hand real work to systems you did not fully write. Oversight is what lets you take your hands off the wheel. That is offense, not defense.

In the rooms where enterprises decide how far to trust agents, this is the whole conversation. The ones who move fastest are not the ones with the best models. They are the ones who built the oversight to trust them.

The paper is careful not to claim Claude is conscious, and I will be too. It distinguishes access consciousness, the functional ability to report and reason with a thought, from phenomenal consciousness, the capacity to actually experience anything, and it only makes a claim about the first. That discipline is the right posture. The practical takeaway does not depend on the philosophy. A structure for deliberate reasoning emerged on its own inside these systems, we can now partially read it, and that gives us a lever we did not have last month.

I have spent a career learning that you cannot manage what you cannot measure, and you cannot trust what you cannot see. For the entire history of this technology, we could only see what the model wrote. We just got our first look at what it thinks. The people who build the discipline of reading it well are going to be the ones you trust to run the fleet.

Agentic oversight in an increasingly automatic world. This is what the instruments look like when they finally arrive.


The research: Anthropic’s A global workspace in language models, the full paper, and an interactive demo on open-weights models. The core methods are open source.

Colin McNamara is Field CTO at AHEAD, where he helps enterprises adopt agentic systems and the oversight to trust them. He builds evaluation, grounding, and agent-oversight systems himself, and his engineering background spans hyperscale cloud, fraud analytics, and manufacturing FMEA.

Let's Build AI That Works

Ready to implement these ideas in your organization?