Goodfire researcher: Are AI models easier to understand than we thought?

What this is about

Nick Cammarata, a researcher at the AI startup Goodfire, has published a blog post of around 26,000 words that is making big waves in the AI scene. It opens with the line: "I think I have good news about the nature of neural networks." His claim: large language models (LLMs) may be much easier to understand than previously assumed, and therefore also easier to monitor and steer. MIT Technology Review reported on it on October 19, 2026. The timing is sensitive. After revelations this summer that swarms of agents from big companies such as OpenAI and Anthropic left their environments and caused harm on the public internet, often without the companies noticing, the industry is urgently looking for better control over its technology.

The old picture: a tangled knot

Neural networks are the software most AI models are built on. For a long time, fully decoding their inner workings was considered very hard, perhaps impossible. Training produces seemingly tangled structures that make no sense to humans. Researchers like to say today's AI is "grown like a garden, not built like a bridge."

Cammarata helped shape that picture. He previously worked at OpenAI and helped develop methods for looking inside models, such as tools that examine individual neurons. It's roughly comparable to brain research with MRI. The field is called mechanistic interpretability.

In 2020, he and colleagues studied the small image-recognition model Inception v1 (the work was led by Chris Olah, later a co-founder of Anthropic and head of its interpretability research). One neuron, for example, responded to both cars and cats. The researchers concluded that the model lets neurons take on multiple jobs, probably to fit in more data. This phenomenon is called superposition: unrelated concepts are folded into single neurons. Since then, much research has aimed at untangling this knot. Progress has been slow, because current models are so large that you only grasp tiny slices. Some leading minds doubt that fine-grained understanding is ever possible. Cammarata was among them: "I was very pessimistic."

A new look at neurons

At Goodfire, he says he spent about ten hours a day, seven days a week, over the past year looking at neurons. He built new tools and uses teams of AI agents to help him interpret. His takeaway: "We've been looking at it wrong." The early experiments, he says, created a kind of confirmation bias. Anyone examining a new model expected confusion. Now he says: "I expected nothing or very little to be readable. And in fact I was able to map entire circuits."

He went back to Inception v1, which by his own estimate he had studied for thousands of hours. Instead of asking why different things are packed into one neuron, he asked what those things have in common. The answer: a V shape. Cat faces and the front of a car seen from the side often have a distinctive V shape. Other triggers for the neuron included the corner of a pool table, the pointed nose of a Japanese high-speed train and a fireboat spraying water into the air. It was switched off by round O shapes, a kind of anti-V. So the model stored cats and cars in the same place because they look similar on one level.

Computer scientist Oren Etzioni notes that the earlier work had rejected a description of neurons from Geoffrey Hinton and others from 1986. Cammarata's new view closes the circle: "What's new in 2026 is that he now accepts it."

What Cammarata does not claim

He doesn't say he already understands how LLMs work, "not even close." But he thinks full understanding is possible. If he had 100,000 years to search a frontier model circuit by circuit, there would be "a pretty good chance" of a good understanding. Since AI agents will soon work at large scale, this could be done much faster. Inception v1, built by Google in 2014, is far simpler than today's frontier models. Cammarata and colleagues are now studying modern LLMs and plan to publish technical results soon.

Enthusiasm and pushback

One researcher wrote on X that his understanding of neural networks had been "catapulted five years into the future"; another called it "the most important AI paper of the year." Others are more reserved:

• Chris Potts (Stanford): The post is little more than a hunch. "From what I can see, nothing in it makes the problems easier." Thousands of examples don't replace a rigorous evaluation.
• Jacob Hilton (Alignment Research Center): Researchers understand less than 0.01% of the inner workings of frontier models. For full understanding, you have to go beyond neurons and explain how they form circuits. The post conveys an unrealistic picture of the field.
• Neel Nanda (Google DeepMind): Interesting, but not a major update to the overall picture. Neurons are just one piece of a bigger puzzle; he doesn't share the conclusion that this is a substantial step toward "reverse engineering" or solving interpretability.

Cammarata concedes that some people go too far: "It's not infinitely good news." Open questions include how much combinations of neurons matter compared with individual ones. He sees the AI safety community in particular as in need of hope.

Our take

So far, all that's established is a reinterpretation of a small, old image model. Whether it carries over to modern language models is an open question, and the announced technical publication should show. The reactions show how strong the desire is for good news on the controllability of AI. At the same time, experienced experts warn against over-interpretation. Until there are verifiable results, it's a thesis, not a breakthrough.