When2Think: Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models
Jaejun Shim, HyunJin Kim, Young Jin Kim, JinYeong Bak
Sungkyunkwan University · Microsoft
Published September 17, 2026 · on the feed September 18, 2026 · ▲ 50 · arXiv 2609.19671
The abstract
Large Reasoning Models (LRMs) achieve strong performance on complex tasks but exhibit systematic inefficiency: they often overthink easy problems and underthink hard ones. Existing approaches based on uniform length penalties or rigid routing incur an efficiency tax, trading reduced computation on easy instances for accuracy loss on hard instances. We formulate efficient reasoning as an instance-adaptive computation allocation problem and propose When2Think, a post-training framework for hybrid reasoning that dynamically allocates computation based on problem difficulty. Our method introduces Instance-level Difficulty-Aware Control (IDAC), a reward-shaping mechanism that leverages pre-computed reference statistics (accuracy and token usage) to regulate reasoning depth. Combined with verifier-based rewards and batch-wise standardized advantages, IDAC enables stable critic-free optimization without learned reward models or online reference-model queries. When2Think encourages direct answering on easy instances while preserving extended reasoning on hard instances, thereby learning when to use System 1 (NoThink) versus System 2 (Think). Experiments on mathematical benchmarks demonstrate improved accuracy-efficiency trade-offs: on AIME24, Pass@3 increases by 10.0% while token usage is reduced by 27.9% relative to the base model, and on AIME25, When2Think achieves 40.0% Pass@3, outperforming compression and routing-only baselines.
The Brief
2:25A short podcast summary of the paper. Written and voiced by AI.
Transcript
When2Think. Learning Difficulty-Aware Length Control for Efficient Hybrid Reasoning Models. By Jaejun Shim and three others, from Sungkyunkwan University and Microsoft. Published on the seventeenth of September. No code repository is linked to it.
Large reasoning models perform strongly on complex tasks but are systematically inefficient. They often overthink easy problems and underthink hard ones. Existing fixes, whether uniform length penalties or rigid routing, pay what the authors call an efficiency tax. They save computation on easy instances, and pay for it with lost accuracy on hard ones. A uniform length penalty treats every problem alike, so it trims the long reasoning that hard problems need along with the excess on easy ones. Rigid routing fixes the choice in a way that cannot bend to the particular instance.
When2Think treats efficient reasoning as a problem of allocating computation per instance. It is a post-training framework for hybrid reasoning models that allocates computation dynamically according to how difficult the problem is. The aim is for the model to learn when to answer directly, System 1 in the authors' framing, called NoThink, and when to reason at length, System 2, called Think.
Its mechanism is Instance-level Difficulty-Aware Control, or IDAC, a reward-shaping method. IDAC draws on reference statistics computed in advance, accuracy and token usage, to regulate how deep the reasoning goes. Combined with verifier-based rewards and advantages standardized across each batch, it supports stable optimization with no critic, no learned reward model, and no queries to a reference model during training. Put simply, the reference statistics tell the reward how hard a given problem tends to be and how many tokens it usually takes, so the shaping can push toward shorter answers where that is safe and leave room where it is not.
On mathematical benchmarks, the accuracy-efficiency trade-off improves. On AIME24, Pass@3 rises by ten percent while token usage falls by twenty-seven point nine percent, relative to the base model. Both sides of the trade-off move the right way at once: more accuracy, fewer tokens. On AIME25, When2Think reaches forty percent Pass@3, outperforming baselines that only compress or only route.
The difference from a uniform penalty is that the penalty now knows what it is looking at. Easy problems are pushed toward a direct answer, and hard ones keep their extended reasoning, so the savings come from where thinking was not needed.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here it is a model that thinks too much and too little at once.
You run a reasoning model on maths problems. On easy questions it writes long reasoning it does not need; on hard ones it sometimes stops too early or loses the thread.
The road ahead
- 1 · The problem ~7 min Reasoning models spend thinking in the wrong places: too much on easy maths problems, too little on hard ones, and blunt fixes trade one for the other.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min When2Think rewards each correct answer with an efficiency bonus scaled by how hard that problem usually is and how long it usually takes.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Pre-compute per-problem statistics, sample Think or NoThink answers, verify them, shape rewards with IDAC, then standardize advantages batch-wise and update.
- 6 · The experiment ~6 min You’ll guess: On AIME24, relative to the base model, by how much did When2Think cut token usage?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.