Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling
Panagiotis Theodoropoulos, Nan Jiang, Xintong Duan, Ali Hasan, Yuriy Nevmyvaka, Evangelos A. Theodorou, Wei Deng
Georgia Institute of Technology · University of Texas at El Paso · Morgan Stanley
Published September 29, 2026 · on the feed October 2, 2026 · ▲ 9 · arXiv 2609.38104
The abstract
Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models (LLMs). High-probability sequences are amplified under the base model without parameter updates or external rewards, avoiding the costly optimization and jagged generalization of RL. However, this approach faces a fundamental exploration--exploitation trade-off, as % strong sharpening restricts exploration, trapping samplers in plausible but incorrect reasoning trajectories, whereas weak sharpening leaves the answer distribution diffuse. To resolve this trade-off, we introduce Parallel Power Tempering (PPT), instantiating power-sharpened LLM sampling via parallel tempering. Running multiple interacting replicas in parallel at different sharpening levels allows lower-power replicas to explore diverse reasoning trajectories and higher-power chains to further exploit higher-likelihood responses favored by the sharpened target. Specifically, we tailor to inference-time sampling by mitigating a truncation bias, identified in prior power samplers, and investigate effective swap strategies under finite memory and compute budgets. Extensive experimentation shows that substantially improves single-chain power-sharpened sampling and outperforms RL-post-trained models, producing higher-quality reasoning traces and even achieving performance comparable to frontier models.
The Brief
2:19A short podcast summary of the paper. Written and voiced by AI.
Transcript
Explore Broadly, Reason Sharply. Push Small Models toward the Frontier via Sampling. By Panagiotis Theodoropoulos and six others, from the Georgia Institute of Technology, the University of Texas at El Paso, and one other institution. Published on the twenty-ninth of September. No code repository is linked to it.
Reinforcement-learning post-training is the usual route to better reasoning in large language models, and it is costly, with generalization the authors call jagged. Power-sharpened sampling offers an inference-time alternative. It amplifies the high-probability sequences the base model already favours, with no parameter updates and no external reward. That alternative has a trade-off built into it. Sharpen strongly, and exploration narrows, so the sampler gets trapped in reasoning trajectories that look plausible but are wrong. Sharpen weakly, and the answer distribution stays diffuse. One dial cannot serve both needs at once.
Parallel Power Tempering, or PPT, stops asking a single dial to do both jobs. It instantiates power-sharpened sampling through parallel tempering: several interacting replicas run side by side, each at a different sharpening level. Lower-power replicas are free to wander across diverse reasoning trajectories. Higher-power chains concentrate on exploiting the higher-likelihood responses that the sharpened target favours. Exploration and exploitation are split across replicas rather than traded off inside one.
Adapting the technique to language model inference takes two further pieces of work. The first addresses a truncation bias that prior power samplers were identified as having, and the method is tailored to mitigate it. The second is an investigation of swap strategies, meaning how the replicas exchange with one another, under finite memory and compute budgets, since the replicas are not free to run.
According to extensive experiments, PPT substantially improves on single-chain power-sharpened sampling. It also outperforms models that were post-trained with reinforcement learning, produces higher-quality reasoning traces, and in some cases reaches performance comparable to frontier models. All of this comes from sampling alone, with the base model's weights left as they were.
Read against the title, the claim is that a small model's reach may be limited less by its weights than by how it is sampled. Run several levels of sharpening together, let them swap, and the exploration that strong sharpening throws away can be kept.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
One dial, two jobs. That is the wall this quest starts at.
You want a small language model to reason better without retraining it. Sampling from its own distribution sharpened toward high-probability answers helps, with no reward and no weight updates.
The road ahead
- 1 · The problem ~6 min Sharpening a small model's own answers toward likely ones helps it reason, but a single sharpening dial cannot both explore and focus.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Run several chains at different sharpening levels side by side, and let neighbours swap responses so good paths reach the sharpest chain.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Set a ladder of sharpening levels, refine each replica in parallel, swap responses between neighbours, and return the sharpest replica's answer.
- 6 · The experiment ~6 min You’ll guess: Using sampling alone, with no weight updates, how did the method compare with models post-trained by reinforcement learning?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.