Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

Diminishing Returns of Data in Sparse Autoencoders. Functional Behavior does not Guarantee Feature Reproducibility. By Sambit Ghosh and three others, from IBM, the Indian Institute of Science, and one other institution. Published on the fourteenth of October. No code repository is linked to it.

Sparse autoencoders, or SAEs, are typically trained on hundreds of millions to billions of activation tokens. Little work has tested whether that scale is actually necessary. The answer here depends on what you want the SAE for. If the goal is good functional behavior, reconstruction in particular, far less data often gets close to the full-data result. If the goal is learned features that reproduce one to one, more data keeps helping. Functional behavior, as the title puts it, does not guarantee feature reproducibility.

The study trains four SAE architectures on three language models, with data budgets from five million tokens up to a reference of five hundred million. Two notions of convergence are measured: functional behavior on standard evaluations, and one-to-one reproducibility of the learned features.

Functional data requirements turn out to be strongly metric dependent. Reconstruction often approaches the full-data reference at substantially smaller budgets, while feature reproducibility continues to improve with data. One concrete case: for Gemma-2-2B with the BatchTopK architecture, the share of strictly matched features rises from two percent at five million tokens to twenty-nine percent at fifty million. How the data is chosen matters too. Random sampling better preserves the original activation distribution and generally favors reconstruction, whereas diversity-based sampling frequently improves AutoInterp.

A theoretical analysis backs the split. It separates statistical consistency of training on a subsample from the stronger conditions, identifiability and optimization, that one-to-one feature recovery requires. Getting the first does not deliver the second.

The claim is about matching the budget to the goal, not about a single number of tokens that suffices. The savings are often available, not always, and they apply to functional metrics, with reconstruction the clearest case. Feature-level claims are where the caution sits: a feature found in one run should not be assumed to reappear in another without an explicit check. And the evidence covers four architectures on three language models.

The practical upshot the authors draw is twofold. SAE training can often be made far more data efficient when the budget is chosen for the goal at hand, and anything said about individual features should be backed by an explicit reproducibility check. Together, they argue, that points toward SAE research that is both cheaper and more reliable.