Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
Jinyuan Li, Chengsong Huang, Langlin Huang, Donghong Cai, Shiping Gao, Yuyi Yang, Jiaxin Huang
Washington University in St. Louis · University of Michigan, Ann Arbor
Published October 3, 2026 · on the feed October 8, 2026 · ▲ 71 · arXiv 2610.04299
The abstract
Self-evolving reasoning models learn from their own generated questions, yet repeated self-training can lead to performance collapse. In this paper, we investigate why performance deteriorates over successive rounds and how to sustain self-evolution. Our analysis identifies two recurring quality problems in self-generated questions: invalid questions and repeated variants of the same mathematical questions. First, invalid questions become more prevalent across rounds, and answer-consistency filtering further increases their proportion in training data. Second, existing question diversity controls based on lexical similarity can miss mathematically equivalent questions expressed in different ways, which leads to question diversity collapse in later training rounds. Building on these findings, we introduce R-Quest, which uses question validity and novelty feedback to guide self-evolution. We first train the solver to recognize and reject invalid questions, then use its judgments to guide questioner rewards and filter solver training data. To avoid question repetition, we use a frozen base model to compare sampled question pairs and provide novelty feedback. Empirically, our method consistently achieves the highest average performance on 12 benchmarks in mathematical reasoning, general-domain reasoning, and code generation across two model families. Additionally, R-Quest maintains stable performance gains over ten rounds of self-evolution, peaking in the final round and outperforming R-Zero by 17.32 points.
The Brief
2:31A short podcast summary of the paper. Written and voiced by AI.
Transcript
Questioning the Questions. Sustaining Self-Evolution in Reasoning Models. By Jinyuan Li and six others, from Washington University in Saint Louis and the University of Michigan, Ann Arbor. Published on the third of October. No code repository is linked to it.
Self-evolving reasoning models learn from questions they generate themselves. The trouble is that repeated rounds of self-training can end in performance collapse. The authors set out to find why performance deteriorates round after round, and their analysis points at the questions. Two quality problems keep recurring. Some generated questions are invalid. Others are repeated variants of the same mathematical question, dressed differently.
Each problem has a twist that makes it worse. Invalid questions grow more common across rounds, and answer-consistency filtering, meant to clean the data, actually raises their share of the training set. Repetition slips through for a different reason. Existing diversity controls judge similarity lexically, by the words, so they can miss questions that are mathematically equivalent but phrased another way. In later rounds, the result is a collapse in question diversity.
R-Quest answers both problems with feedback on the questions themselves. One signal is validity: is this a real question. The other is novelty: is it genuinely new, or a familiar question in fresh words. Those two signals guide the self-evolution process. The setup still has a questioner that writes problems and a solver that learns from them; what changes is that the questions now face scrutiny of their own before they shape either side's training.
For validity, the solver is first trained to recognize and reject invalid questions. Its judgments are then put to two uses: shaping the rewards given to the questioner, and filtering the data the solver itself trains on. For novelty, a frozen base model compares sampled pairs of questions and supplies feedback on whether they repeat each other, which sidesteps the weakness of purely lexical comparison.
Across twelve benchmarks in mathematical reasoning, general-domain reasoning and code generation, and across two model families, the method consistently posts the highest average performance. It also holds up over time. Through ten rounds of self-evolution, R-Quest keeps stable gains, peaks in the final round, and finishes ahead of R-Zero by seventeen point three two points.
The diagnosis carries the paper. The collapse is traced to the questions the model learns from, which degrade in two specific ways, one of them made worse by a filter meant to help. Checking the questions for validity and novelty is what lets the gains keep building to the last round instead of fading.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free