StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?

StudyBench. Can Self-Evolution Squeeze Textbooks for Olympiad Capability?. By Yinghao Chen and nine others, from Tsinghua University and Zhejiang University. Published on the first of September. Its code repository is public and real: about thirty source files.

A person can study a handful of well-written textbooks, master a discipline, and go on to attempt its hardest problems. The authors argue that an ideal self-evolution method for language models should share that property: it should learn autonomously from raw training material and come away with problem-solving capability that transfers. The difficulty is that nothing measures this directly. There has been no clean way to ask how much capability a method actually extracts from the material it is given, and without that measurement, claims about self-evolution are hard to compare or to pin down.

StudyBench is built to ask exactly that question. It is a controlled physics benchmark, and what it measures is efficiency of conversion: how well a self- evolution method turns training material into capability. The material is textbooks. The question in the title is whether self-evolution can squeeze those textbooks hard enough to reach olympiad capability, the level at which the discipline's hardest problems sit.

The test set is organised in two parts. The Application Set holds difficult textbook problems, and it evaluates absorption, whether the method took in what the material teaches. The Transfer Set holds olympiad-level problems, and it evaluates transfer, whether that absorbed knowledge carries over to much harder ground. The two sets separate learning the material from being able to use it on unfamiliar problems. A guidance ablation then adds a reference point: the same material is supplied directly as in-context guidance, which shows how much that material can unlock when a model is simply handed it at the moment of solving.

Representative self-evolution methods were benchmarked across three base models, and three results stand out. First, improvements on the Application Set rarely translated into gains on the harder Transfer Set. Second, the guidance ablation exposes what the authors call a Guidance Gap: even the strongest method closes only a small fraction of what the same material unlocks when supplied as in-context guidance. Third, every method hits a Compute Plateau, saturating well before it has exhausted its compute budget.

Together those findings point in one direction. The material demonstrably contains more than the methods extract, because guidance shows it. More compute does not close the difference, because the methods plateau before running out. So the remaining gap is a method problem, not a data problem and not a compute problem.

The benchmark reframes what progress in self-evolution should look like. Rather than an open-ended pursuit judged by whatever improves, it becomes a measurable target: how much of a textbook's value a method can capture, with in-context guidance marking how much value was there to capture in the first place. The code is released alongside it.