Learning Self-Orchestration in Decentralized Agent Swarms

Learning Self-Orchestration in Decentralized Agent Swarms. By Guowei Xu and four others, from Harvard University. Published on the fifteenth of October. Its code repository is worked out: about three hundred source files, with a readme and a licence.

How capable an agent swarm is depends on two things: its individual members, and how they coordinate. Existing multi-agent systems often train agents inside fixed orchestration structures. That separates the learning of individual capabilities from the organisation of collective behaviour. What each agent can do gets learned, while how the group arranges itself is decided in advance and stays put.

Market-based post-training, or MPT, learns both at once: action policies, and self-orchestration, in a decentralized swarm. The organising device is a bid. Agents express how willing they are to participate by bidding, and those bids determine who acts, and how collective outcomes are turned into local training credit. Because acting and bidding are both learned, individual capabilities and patterns of participation adapt together, and no central coordinating agent decides for them.

The essential point is that a bid plays two roles. It is a selection signal, settling which agent acts. It is also a credit signal, shaping how the result achieved by the group flows back to each member as something that member can learn from. Training adjusts the action policy and the bidding behaviour together, so a change in one is met by a change in the other. What each agent can do and how often it takes part are learned side by side, not one after the other. Nothing outside the swarm assigns the roles. Who acts is settled by the bids, and the bids are themselves something the agents learn.

MPT is evaluated in three settings: embodied crafting, robot navigation, and deep research. It shows gains over monolithic post-training, and agent participation does evolve as learning proceeds. Holding the post-trained action policies fixed, learned self-orchestration outperforms alternative coordination strategies. Ablations show that both action and bidding benefit from being learned, and that learned bids provide more effective signals for training the actions.

The effects carry past the original training objectives. Trained swarms adapt faster to related tasks, and perform better when the population is larger at inference time. Larger populations also lose less performance when agents drop out.

Coordination, on this account, is not a structure laid over the agents from outside. It is something they learn alongside their skills, expressed through the bids they place. The authors draw three consequences from that: the swarms can be reused, they scale, and they are robust when members go missing. Code and trained models are released.