ProgAgent: Learning Tool Orchestration through Programmatic Training

ProgAgent. Learning Tool Orchestration through Programmatic Training. By Hao Li and six others, from the Hong Kong University of Science and Technology, the Beijing University of Posts and Telecommunications, and one other institution. Published on the tenth of October. Its code repository is worked out: about two thousand five hundred source files, with a readme and a licence.

Large language models are increasingly expected to orchestrate many different tools over long-horizon tasks. Most agents do this through Direct Tool Calling: the model invokes tools one after another, and every tool response piles up in its context. Over a long task, that accumulation can lead to context rot and to orchestration breaking down. Programmatic Tool Calling is the alternative. Tools are invoked through executable code, and intermediate results are kept in runtime variables instead of being appended to the context. Prior work has explored this mainly as an inference-time optimization. Whether it helps when training agents has remained unclear, and verifiable training tasks suited to it are scarce.

ProgAgent is a full training stack for programmatic tool calling. Rather than treating code-based calls as a trick applied at inference, it trains models to work in that interface, and it supplies the verifiable tasks needed to do so.

The stack has three parts. A pipeline synthesizes verifiable long-horizon tasks. A lightweight runtime executes the programmatic calls. And a two-stage training recipe does the teaching. The data requirement is modest: only two hundred and fifteen tasks, each instantiated at four environment scales.

Across four benchmarks, ProgAgent reaches higher pass rates than the evaluated baselines of comparable size. The sharpest result is on LOCA-bench. At an environment state of sixty-four thousand tokens, ProgAgent-14B achieves a thirty point seven percent pass rate, where those baselines score zero.

Cross-harness experiments then swap the evaluation interface. Models trained with programmatic calling outperform models trained with direct calling under both interfaces, which suggests that the benefit of this training extends beyond the interface itself.

The question underneath is where intermediate results should live. Prior work treated keeping them in variables as a way to run an agent at inference time. ProgAgent treats it as something to train for, and the cross-harness result hints that what a model learns this way carries over even when it is handed the older, direct interface.