Neural Networks Are Already Clean (Nick Cammarata)

An interactive essay by Nick Cammarata, published on Goodfire’s site in October 2026 and subtitled “A working hypothesis about neurons, generalization, and the ecology of circuits.” A technical companion paper is promised. Cammarata co-wrote several of the early interpretability papers this essay now revisits, so it is partly a correction of his own earlier work.

The argument

1. Networks run on one learning principle, not two. The field explained “polysemantic” neurons (one neuron, several unrelated jobs) with superposition: a network short of neurons folds unrelated concepts together because they rarely collide, so you have to untangle it with a second model (e.g. a sparse autoencoder) before studying it. Cammarata argues the evidence for this was misread. What looked like arbitrary packing is mostly ordinary generalization.
2. The canonical examples were errors. The famous InceptionV1 “cat faces and car fronts” neuron is a V-shape neuron: it also fires on V-shaped flames, water jets and fox faces, and its negative pole is a round “O”. The one published “superposition circuit” (a car feature apparently hidden among dog neurons, which he found in 2019) was cars and dogs grouped by shared pose, plus a feature-visualization artefact that drifted toward dogs. To his knowledge no one has published a second circuit of that shape.
3. InceptionV1’s unreadable last layer was an architecture problem, not a learning principle. Residual connections and LayerNorm at the same size and data take that layer from roughly 0–5% interpretable neurons to 99–100%. A network that is short of neurons doesn’t fold unrelated things together; its neurons just get coarser (one “car” neuron instead of many). He trained ConvNeXts across a 300× size range and found them legible at every scale.
4. “Recursive generalization” explains most of what is left. A neuron grows offshoots as it learns, the way a branch sprouts sub-branches: an “iy” neuron grows into “diy” and then “do it yourself,” which contains no “iy” at all. Each step makes sense locally, even though the end result no longer looks “monosemantic.”
5. Language models are far more legible than believed. Anthropic’s 2023 Towards Monosemanticity scored the median neuron 0 out of 14. His own 2023 OpenAI project found about 1.7% interpretable. In preliminary audits of Qwen3-32B today, roughly 90% of neuron poles have one main job, about 7% have two related jobs, and only about 2–4% are unexplained, with similar results up to trillion-parameter models.
6. Understanding compounds. Interpretability works like a crossword: each neuron you explain constrains its neighbours, so a nearly empty map and a nearly full map are both stable states. Bigger models help, because their neurons are narrower. Cases that looked like noise kept resolving: a “pr” neuron also fired on Base64 URLs and Bengali, which decode or translate to “pr”, and a “comma” neuron turned out to track Colorado.
7. Superposition should be the last hypothesis, not the null. Its only symptom is that something doesn’t make sense, and bad tools, bad architecture or unfollowed generalization produce the same symptom. Each of those can be tested. Superposition can’t.

Key takeaways

• Neurons are better read as verbs than nouns. A neuron is less about what it sees than about what it routes. It separates two patterns at its positive and negative poles and acts as a gate for the neurons downstream. Ask “how would I describe this without listing nouns?” and the shared pattern usually appears.
• The negative pole is half the explanation, and a quick check on whether a grouping is real.
• A “bitter lesson” for interpretability: scaling ordinary investigation with agents beats engineering clever decompositions — the same pattern Rich Sutton described for AI as a whole.
• Most speculative claim, flagged as such by the author: this may be most of what is needed, and interpretability may be far more tractable, and sooner, than the field has assumed.

Where it fits

This is the strongest optimistic case in the corpus that we will be able to see inside the systems we are building, and interpretability sits upstream of AI Alignment and AI Safety. Weigh it as one researcher’s working hypothesis from a company that sells interpretability, with the detailed paper still to come. Read it next to Penguin has moved up the… (tweet), where Geoffrey Hinton’s forthcoming book makes the case about risk.