Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents

Mid-Harness. Scaling Actions Between Model and Harness for Terminal Agents. By Minki Kang and ten others, from NVIDIA and KAIST. Published on the thirtieth of September. No code repository is linked to it.

A terminal agent acts through stochastic model generations, one command at a time. Being able to generate a useful action is not the same as reliably executing one. A single poor command, a wrong package install for example, can change the environment in ways that make every later step harder. And this can happen even when the same model was perfectly capable of generating a better alternative. The good action was available. It just was not the one that got run.

The question the authors ask is whether test-time compute, spent at the boundary between the model and its harness, can make actions more reliable and whole trajectories more successful, and what makes that spending effective. Their instrument is Mid-Harness. It sits between the generator and the harness, samples several candidate actions, verifies them, and forwards only one for execution. The generator is left unchanged, and so is the harness. Only the step in between is new.

Once the sampling is in place, the verifier turns out to be the deciding piece. With a TMAX nine B generator, drawing more candidate actions brings little benefit when verification is weak. A capable verifier, on the other hand, can find and exploit the useful alternatives that the very same generator was already producing. When the TMAX model itself serves as the verifier, pairwise verification performs best among the mechanisms the authors evaluated. They also distil responses from a stronger verifier into TMAX, which further improves results, again without touching the action generator.

On TerminalBench-Lite, the base agent scores fifty percent at Pass at one. With a GPT five point six Sol verifier choosing among eight sampled actions, that rises to sixty-eight point zero three percent.

Cost is part of the picture too. With TMAX on TerminalBench-Lite, combining action scaling with trajectory scaling reaches higher success at a lower estimated token cost than simply generating more trajectories on their own. Beyond that setting, Mid-Harness also improves performance across additional models, benchmarks and harnesses.

Test-time compute for agents is usually pictured as more attempts at the whole task. This work points at a smaller unit. Spend the extra compute one action at a time, before the command touches the environment, and let a good verifier pick from what the model can already generate. The authors name action scaling as a promising target for test-time compute in terminal agents, and the evidence here suggests the verifier, more than the sample count, is where that promise lives.