Sharpening Tax in Post-Training

Sharpening Tax in Post-Training. By Changdae Oh and nine others, from Meta Superintelligence Labs, the University of Wisconsin–Madison, and two other institutions. Published on the first of October. Its code repository is public and real: about twenty-five source files.

An emerging hypothesis about reinforcement learning post-training of large language models holds that it merely sharpens behaviours the base model already has. Single-shot accuracy improves, and solution coverage is the price. That trade-off has been observed in math and coding. It need not carry over to agentic tasks, where multi-turn tool use and interaction might require capabilities newly acquired during post-training. The authors' finding, which they call surprising, is that it does carry over. Pre-trained models, given a light inference harness, can serve as capable agents. Their accuracy at pass at one is far lower. Yet given a sufficient test-time budget, they often surpass their post-trained counterparts in solution coverage, measured as pass at K.

The authors also analyse the mechanism. Post-training pushes tasks toward two extremes: always solved, or never solved. That improves sampling efficiency and consistency, and it costs solution coverage. The arithmetic of repeated sampling explains why. A task that is always solved gains nothing from a second attempt, and a task that is never solved gains nothing from any number of further tries. The tasks in between, solved some of the time, are the ones that more samples can rescue, and post-training thins that middle out. To measure that cost, they propose the Sharpening Tax, a diagnostic metric that quantifies the loss in test-time scalability after post-training.

The tax is measured across fourteen pairs of base and post-trained models, from four families, on three agentic benchmarks: forty-two cases in total. Three properties are reported. It is prevalent in most settings. It can be estimated from a few rollouts. And it correlates well with other metrics.

The claim has edges, and they are stated plainly. The tax is prevalent in most settings, not all of them. The headline comparison does not say base models are better agents. The coverage advantage of base models appears given a sufficient test-time budget, and their single-shot accuracy is far lower. The cases are agentic benchmarks, which is where the question was open, rather than math or coding. And the remedy is tested in two agentic environments.

That remedy is posterior-tempered group sampling, or PTGS: a simple, plug-and-play Bayesian sampler that adapts the sampling temperature for each prompt to its estimated difficulty. Applied during RL training, it pays a smaller tax than the fixed-temperature baseline. It solves more tasks under repeated sampling, and it improves single-shot accuracy as well.

Sharpening, on this account, is a real trade with a measurable price, not a free improvement. Post-training buys consistency by polarising tasks into always and never, and the never side is where coverage is lost. Naming the cost turns it into something a training method can be asked to reduce, which is exactly the test PTGS is put to.