Sparse Readout Prism: Explaining Logit-Lens Scores in Features Instead of Tokens

Sparse Readout Prism. Explaining Logit-Lens Scores in Features Instead of Tokens. By Matteo He and three others, from the University of Cambridge. Published on the first of September. Its code repository is worked out: about ninety source files, with a readme and a licence.

A language model's prediction of its next token develops across its layers, and lens methods follow that development by decoding intermediate hidden states into tokens. But what a lens reports reflects two things: the hidden state, and the readout, meaning the unembedding matrix used to decode it. Many lenses are fit on a corpus. The authors show that two lenses differing only in the corpus they were fit on can report different tokens for the very same hidden states. They call this dependence corpus conditionality. A reading that changes with the corpus is partly a reading of the corpus.

Sparse Readout Prism, or SRP, examines the structure of the readout without any fitting corpus at all. It decomposes the readout using only its weights, and expresses any token logit, or any difference between two logits, as a sum of contributions from sparse readout features. As a prism splits light, a single score is broken into the parts that add up to it.

That decomposition makes readout features a new unit of analysis for lens readings. They expose structure that token identities can obscure, and they allow comparisons across tokens, across contexts, across layers, and across different lenses. Instead of asking which token a layer points to, one can ask which features are contributing to the score.

Replace the original readout with SRP's sparse approximation, and it reconstructs between eight point nine and seventeen point three percentage points more of the tested logit differences than the strongest of six baselines. Those baselines are built on geometric relations among the rows of the readout. So the comparison is between two ways of finding structure in the same matrix: relations among its rows, against a sparse set of features drawn from its weights. On the logit differences tested, the features recover more.

A causal check points the same way: ablating features shifts logit differences in proportion to their SRP contributions. Then the corpus question comes back. Token readings vary with the fitting corpus, but the dominant readout feature remains stable across them. The surface of the reading moves with the corpus. What sits underneath it, at the level of features, holds still.

That stability is what gives SRP a role beyond its own analyses. Because no corpus goes into its construction, it offers lens studies a control that is independent of the fitting corpus. Where two lenses report different tokens for the same hidden state, a reading in features gives a reference point that neither fitting corpus had a hand in shaping.