Mental-Models for Multi-Agent Systems

Mental-Models for Multi-Agent Systems. By Hanan Gani and two others, from the University of California, San Diego. Published on the eighth of October. Its code repository is worked out: about one hundred and sixty source files, with a readme.

Large foundation models have sped up progress toward general-purpose agents, ones that deal with humans and with other agents through language and multimodal signals. Deciding well among other agents, though, means reasoning about what they know, what they intend, and what they are likely to do, often under partial observability. Current agentic systems mostly work through prompt design, through memory, or through end-to-end shaping of behavior. What they typically lack is a learned, explicit representation of the partner's state, one that could be reused as a decision variable from task to task. Whatever such a system understands about the other party stays folded into its prompts or its behavior, rather than held as something it can consult when deciding.

The proposal is mental-model-enabled agents. An agent is given a latent mental model of its counterpart. From the history it has observed, it infers that counterpart's hidden beliefs, intentions and likely reactions, and it uses those inferences to choose what to do. The partner's state becomes an explicit quantity: estimated from what has happened so far, and then put to work in the decision about what happens next. That is the missing decision variable the problem described.

Three pieces are learned. The first is an amortized, recursive Theory-of-Mind representation with first-order and second-order mental-state structure. In plain terms, it covers what the partner thinks, and also what the partner thinks about the agent. The second, trained jointly with it, is a belief-conditioned reward model, which scores candidate actions against the partner state that has been inferred. Finally, a policy is learned under that belief-aware signal. The resulting agent can act on its own at inference time and still keeps the benefit of having modeled its partner explicitly.

The same framework is evaluated on two kinds of benchmark: language-only and multimodal. Across those settings, explicit modeling of mental states consistently improves the quality of interaction, and improves Theory-of-Mind performance, compared with the base agentic systems. Two things are being measured there. One is how well the interaction goes. The other is how well the agent reasons about minds. Both move in the same direction.

The authors read this as evidence for an inductive bias. An agent that keeps a structured model of the other party, and lets that model shape its rewards and its choices, does better in multi-agent settings than one that leaves the partner implicit. The same structure held up whether the interaction ran through language alone or through several modalities. What carries over is not a prompt or a stored memory, but a learned picture of the partner that the agent can put to use from one task to the next.