The Geometry of Inference in Transformer Residual Streams

The Geometry of Inference in Transformer Residual Streams. By Timur Mudarisov and two others, from the University of Luxembourg and the London Institute for Mathematical Sciences. Published on the twenty-ninth of September. No code repository is linked to it.

A transformer language model builds its prediction through a series of residual updates. How its representation becomes specific to one eventual outcome has remained unclear. The approach here compares each intermediate residual state with two things: its own final state, and an empirical bank of final states taken from other contexts. One asks whether the state is heading toward where it will actually end up. The other asks how many places it could still plausibly be heading instead. The central claim is that specificity grows in a way that a single distance cannot capture. Distance, the number of competing endpoints, and how concentrated the surviving endpoints are each give a distinct view of the process.

Across six pretrained language models, a state's own endpoint becomes preferable to the average alternative early on, while many individual alternative endpoints remain closer. Those competing sets generally shrink with depth. Their membership changes, though, and the endpoints that survive need not grow more similar to one another. So directional alignment and the rank of the true endpoint can improve while the Euclidean distance to the final state barely changes. The state can be pointing more clearly at its destination without getting much nearer to it.

Two formal pieces support this. A simple high-dimensional model separates the roles of norm, alignment and endpoint geometry, and shows how gradual changes in direction can produce sharp drops in competition. A proof shows that a straight path toward a state's own endpoint cannot introduce new competitors, under either Euclidean or cosine distance. When new competitors are observed entering, the path has therefore departed from straight-line convergence. Finally, endpoints tied to lower-ranked output tokens tend to lie farther away in cosine distance in every model studied, linking residual geometry to how outputs are organized.

The empirical results come from six pretrained models, and the comparison bank is built from final states of other contexts, so competition is measured against that bank. The high-dimensional model is described as simple, and the straight-path result is a statement about what a straight path cannot do, which is what makes observed new competitors informative.

Watching only the distance to the final answer would suggest little is happening across many layers. Counting competitors and checking direction tell a different story, one of sharpening specificity, and the authors give both the measurements and the geometry to explain why the views disagree.