Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge
Chanuk Lee, Minki Kang, Sangwoo Park, Woongyeong Yeo, Jinheon Baek, Sung Ju Hwang
KAIST · DeepAuto.ai
Published September 28, 2026 · on the feed September 29, 2026 · ▲ 40 · arXiv 2609.34327
The abstract
Scaling test-time computation is a powerful way to improve language-model reasoning, and is particularly appealing for small reasoning models (sRMs) that are cheap to serve. However, is additional thinking always the right operation? By intervening at intermediate reasoning states across two model families and multiple scales, we find that self-refinement largely consolidates probability mass onto solutions already reachable from the current state, rather than making new ones reachable. These interventions reveal two failure regimes: execution bottlenecks, where the correct path is reachable and reflection can recover it, and knowledge bottlenecks, where relevant external information makes it reachable. Motivated by this distinction, we introduce FlyBy, a selective querying framework, and train 4B and 8B variants to reason first, diagnose what remains unresolved, and, at a knowledge bottleneck, query stronger models whose parametric knowledge extends beyond its own. Supervised fine-tuning bootstraps a multi-depth query action, and cost-aware reinforcement learning calibrates whether to query, what to ask, and how much to spend. On 1,158 hard problems across six benchmarks, FlyBy-4B achieves 45.96% pass@8, surpassing Qwen3-14B (41.64%) at 2.7 times lower serving cost, while also exceeding Qwen3-8B in pass@1 (16.85% vs. 15.31%). Scaling to FlyBy-8B further improves pass@8 to 51.81%.
The Brief
2:20A short podcast summary of the paper. Written and voiced by AI.
Transcript
Knowing When Thinking Is Not Enough. Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge. By Chanuk Lee and five others, from KAIST and DeepAuto A I. Published on the twenty-eighth of September. Its code repository is worked out: about four hundred source files, with a readme and a licence.
Scaling test-time computation is a powerful way to improve reasoning, and it is especially appealing for small reasoning models, which are cheap to serve. The question here is whether more thinking is always the right move. The authors intervene at intermediate reasoning states, across two model families and multiple scales, and find that self-refinement mostly consolidates probability mass onto solutions already reachable from the current state. It rarely makes new ones reachable. Those interventions separate two failure regimes. In an execution bottleneck, the correct path is reachable and reflection can recover it. In a knowledge bottleneck, the path becomes reachable only once relevant external information arrives.
FlyBy is a selective querying framework built on that distinction. A small model reasons first, diagnoses what remains unresolved, and, when it hits a knowledge bottleneck, queries stronger models whose parametric knowledge extends beyond its own. The authors train a four-billion-parameter variant and an eight-billion-parameter one.
Training has two stages. Supervised fine-tuning bootstraps a query action that can operate at multiple depths. Cost-aware reinforcement learning then calibrates three decisions: whether to query at all, what to ask, and how much to spend. Asking for help becomes an action with a price attached, one the model learns to take selectively rather than by reflex.
Evaluation covers one thousand one hundred and fifty-eight hard problems drawn from six benchmarks. FlyBy-4B reaches forty-five point nine six percent pass at eight, surpassing Qwen3-14B at forty-one point six four percent, with a serving cost two point seven times lower.
At pass at one it also beats Qwen3-8B, sixteen point eight five percent against fifteen point three one. Moving up to FlyBy-8B lifts pass at eight further, to fifty-one point eight one percent.
Rather than asking a small model to think harder about everything, FlyBy teaches it to tell two kinds of being stuck apart. One gives way to reflection. The other gives way only to knowledge from outside, and a model that recognises which one it faces can spend its budget where it counts.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Thinking longer is the usual cure for a struggling model. Here it stops working.
You run a small reasoning model because it is cheap to serve. When it gets a hard problem wrong, the usual fix is to let it think longer: reflect, revise, try again.
The road ahead
- 1 · The problem ~6 min A small reasoning model often fails not because it thinks too little, but because the fact it needs is not in its head.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Tell fixable slips from missing knowledge; for missing knowledge, ask a stronger model a focused question, and learn under a price when asking pays.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Train the small model in two stages, then at test time it reasons, diagnoses the gap, and queries a helper only for missing knowledge.
- 6 · The experiment ~6 min You’ll guess: On hard reasoning problems, how did FlyBy-4B, a small model that selectively queries stronger models, compare with the larger Qwen3-14B on pass@8?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.