Behavioral Compositionality of Sparse Autoencoder Representations

Behavioral Compositionality of Sparse Autoencoder Representations. By Haotian Ma and two others, from the University of Wisconsin–Madison. Published on the fifteenth of October. Its code repository is public but slight: nine source files.

Sparse autoencoders, or SAEs, represent a model's activations with sparse features that are decoded linearly. Linear decoding, though, says nothing about whether the downstream effects of those features add up. The claim here has two halves. SAE coordinates support behavioral descriptions that are comparatively compositional. But they are not fully additive, and faithful explanations of features acting jointly still need interactions beyond each feature's individual effect.

The tool is SAE Additivity Residuals, or SAR: a measure of the behavioral variation left unexplained above a chosen order of interaction. It comes with a formal guarantee. When feature subsets are sampled independently, population SAR is exactly the normalised energy of the interactions above that order, and it bounds the coalition ordering error of the best approximation at that lower order.

To separate compositionality that belongs to the SAE representation from compositionality created by downstream computation, the learned SAE coordinates are compared with random rotations of the same selected subspace that preserve sums. Across all eighteen settings of model and behavior, the learned coordinates show lower first-order residuals. That is the first half.

The second half: pairwise terms improve held-out coalition ordering in every setting, and they beat models with a matched parameter count that use randomly chosen third-order terms instead. In a combined steering experiment, modeling pairs raises rank fidelity from zero point five eight seven four to zero point seven seven nine zero. Ordering error falls from zero point one four four four to zero point zero six zero six.

The exact link between SAR and interaction energy holds under one sampling assumption, independent sampling of feature subsets. The comparative claim is relative to rotated coordinates, not an absolute verdict of additivity. And the empirical picture covers eighteen model and behavior settings.

Both halves of the result matter for anyone using SAE features to explain or steer a model. The learned coordinates are a better basis for additive description than a rotated one. Yet when features are switched on together, explanations and interventions that stay faithful still have to account for how pairs of features interact.