Are Causal Neuron Circuits Unique and Transferable?

Are Causal Neuron Circuits Unique and Transferable?. By Ömer Tekin Yavuz, from Iletisim Yazilim. Published on the tenth of November.

Interpretability work often makes its case with a demonstration: intervene on a group of neurons, and the model's answer changes. One recent example removes an eighteen-neuron group in Qwen3-32B, and the model answers Simba instead of Scar to a two-hop Lion King riddle. Such a result does establish that the group matters. It does not establish how much of the group is necessary, whether some other group would have done the same job, or whether the effect survives a change of input. Those are the three questions this paper puts to the demonstrations, and the headline answer is that the full group is rarely necessary.

The method is exhaustive at small scale. For every selected group, all four thousand and ninety-six subsets of twelve MLP neurons are measured. On a two-alternative capital task in Qwen3-0.6B, zero-ablating the twelve selected neurons lowers the answer margin by twenty-seven percent, where random neurons barely move it. But replacing those same neurons with the values they take on other questions also barely moves it. Their activations are nearly constant across questions, and the author reports no evidence that they carry question-specific content.

On two-hop relation chains the intervention is sharper. Patching twelve neurons with activations taken from the first-hop question shifts the answer to the bridge entity, from Scar to Simba. The greedy answer starts with the bridge in nine of eleven chains for Qwen3-1.7B, nine of eleven for Llama-3.2-1B, and eight of ten for Gemma-3-1B, while random neurons never do it. Yet uniqueness fails: a few neurons often suffice, and in many chains disjoint subsets each flip the answer on their own. In an exploratory analysis, that happens most often when the model is less confident. As for transfer, group effects carry over to rephrasings of the same chain, and are much weaker on the other chains tested.

The boundaries are drawn explicitly. This is a study of neuron-level causal interventions and of alternative subsets, and the author states plainly that it does not recover the full connectivity of circuits. The exhaustive sweep covers subsets of twelve neurons, not arbitrary group sizes. The models are small ones, Qwen3, Llama and Gemma at the low end of parameter counts, and the tasks are a two-alternative capital question and a set of two-hop chains. The confidence observation is flagged as exploratory rather than tested.

A flipped answer is weaker evidence than it looks. It shows that a set of neurons is sufficient to move the model, and sufficiency says nothing about necessity, uniqueness or reach. On these tasks, each of those three further questions comes back with an answer that complicates the picture the demonstration first suggested.