SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?

SAEScientist-Bench. Can AI Agents Conduct Autonomous SAE Interpretability Research?. By Yuqiao Tan and three others, from the Institute of Automation, Chinese Academy of Sciences and the University of Chinese Academy of Sciences. Published on the eighth of September. Its code repository is worked out: about seventy-five source files, with a readme.

Research on recursive self-improvement, RSI, has mostly automated the pipelines that train models. The authors argue that reliable autonomous development needs a pillar that is still missing: monitoring and auditing after the fact, to understand what models have learned and to make sure they stay safely aligned. Mechanistic interpretability tools are what could fill that gap, and among them Sparse Autoencoders, SAEs, are a cornerstone, because they isolate interpretable features that can be used to inspect a model and to steer it. If autonomous research is to include auditing, the question is whether an AI agent can do that interpretability work itself.

SAEScientist-Bench asks whether AI agents can act as scientists who use SAE tools to make mechanistic discoveries on their own. Agents are scored on three dimensions: activation rank, concept selectivity on contrastive texts, and causal steering. Each is judged against curated reference features chosen by experts and anchored on Neuronpedia.

Each task gives the agent a target concept. The agent designs contrastive probes and navigates a Gemma Scope dictionary of more than one hundred and thirty-one thousand features in Gemma-2-9B-IT, looking for the single feature that best captures the concept. The agent is cast in the role of the researcher. It chooses its own probes, texts that contrast the concept with its absence, and it has to decide for itself which of a very large number of candidate features is the right one. The evaluation covers ten agent configurations and twenty tasks.

Frontier agents show genuine discovery ability, and different agents lead on different dimensions. All of them, though, remain well behind the expert baseline. They come close to expert level at separating a target concept from contrastive controls, and fall well short on causal steering of generation. Finding a feature that responds to a concept, in other words, is closer to solved than finding one that can reliably push the model's output toward it.

Further analysis looks at where the agents go wrong. Agents can design contrasts that rule out spurious candidate features, which is the experimental design half of the job. But they frequently misinterpret the measurements their experiments produce.

The benchmark turns a vague aspiration into a number. Understanding a model through experiment becomes, in the authors' words, a measurable capability for closed-loop autonomous AI research and development. The early reading is that agents can already set up the right experiments, and still struggle to read what those experiments are telling them. Code is available.