Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents

Trajectory2Gym. Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents. By Yihao Hu and seven others, from Westlake University and Ant International. Published on the twenty-fourth of October. Its code repository is worked out: about ninety-five source files, with a readme and a licence.

A tool-use trajectory that succeeds is not automatically good supervision. A successful interaction may underrepresent the multi-step dependencies an agent needs to learn. It may also contain actions that nothing in the agent's view justified at the moment they were taken. Training on such an action teaches the learner to make a move it had no visible reason to make.

The paper names two coupled requirements for supervision that works. The first is compositional coverage: the important dependencies between observations and actions have to be represented. The second is decision grounding: every action that is supervised has to be supported by the interaction history the learner can see. Trajectory2Gym is built to satisfy both. It reframes trajectory generation as deliberately constructing interaction experience that can be learned from, instead of collecting trajectories that happened to succeed.

It starts from compositional structures that are under-supported and constructs executable interactions around them. Supervision is then derived only from the grounded decisions in the executions that actually result. Two further pieces shape training. Structural-support selection decides how the resulting experience is allocated. And parent-normalised supervised fine-tuning keeps the semantic interaction as the unit that carries training weight.

The order of operations matters. Rather than generating many trajectories and filtering for success afterwards, the method decides first which structures need more support, builds interactions to supply it, and then keeps only the decisions that the visible history can justify. What reaches the learner has been chosen for what it can teach, not for whether the episode ended well.

The evaluation covers five Qwen3 scales and three benchmarks: Tau1, Tau2 and BFCL. Trajectory2Gym improves all three at every scale tested. The equal-weight mean across the three benchmarks rises by between sixteen point nine and twenty-nine point eight percentage points, depending on scale. The gains are largest in stateful and multi-turn interaction, the settings where later decisions depend on earlier observations and feedback.

Where the gains concentrate matches the diagnosis. Long, stateful interactions are where dependencies matter and where an unjustified action is easiest to slip in. The authors' conclusion is that scaling tool-use agents is not only a matter of more successful trajectories, but of interactions whose dependencies are represented and whose decisions can be learned from what the agent can see.