MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

MIMESIS. Learning User Simulators as Training Environments for Interactive Agents. By Hoang Phan and nine others. Published on the seventh of October.

Interactive language agents need users to learn from. Training and evaluating them typically requires rich user interactions, and collecting human feedback at that volume is expensive and hard to scale. Simulated users are the obvious substitute, but a simulator has two jobs. It has to resemble how real users behave, and it has to give the agent a useful learning experience. Most agent-training frameworks hand that role to an off-the-shelf assistant model. The trouble is that an assistant is built to be helpful, and that helpfulness makes it a poor stand-in for a person. Compared with real users, it tends to be overly cooperative, overly explicit, and behaviorally homogeneous.

MIMESIS is a purpose-built user simulator rather than a borrowed assistant. It is trained on human conversations, with explicit reasoning supervision, and with thirteen realistic behavioral patterns derived from real user interactions. The aim is a simulated user that behaves like the people an agent will actually meet, and that can then serve as a training environment.

The work proceeds in stages. First the simulator itself is trained. Then it is frozen, and an agent is trained by interacting with the frozen simulator through multi-turn reinforcement learning. On top of that, the authors propose Coached On-Policy Self-Distillation, or CSD. The simulator produces private reasoning traces alongside its utterances, and CSD treats those traces and the utterances that follow as feedback on how well the agent is addressing the user's needs. A coach turns that information into concise coaching notes, describing how the agent could better anticipate those needs and adapt its behavior as the interaction unfolds. The result is dense, token-level supervision, going beyond the sparse reward a task provides at the end.

The simulator is measured first. The nine billion parameter model reaches a SOUL-Index of sixty-five point seven, surpassing the strongest frontier model. The strongest baseline on RealUserSim and SimulatorArena is Claude-Opus-5. Against it, MIMESIS improves behavioral fidelity by thirteen point four points on the first, and reduces Turing distance by three point six points on the second.

Then the agents trained against it. Across eight environments, an agent trained with MIMESIS performs better than one trained with GPT-5.5 as the simulated user, and that holds under all nine unseen user simulators used for evaluation. The authors read this as stronger generalization to new user simulators. Adding CSD yields further gains across all nine evaluation user models.

The thread running through the paper is that the user in the loop is part of the training signal, not just its delivery mechanism. A user who is too agreeable teaches an agent very little about real people. MIMESIS replaces that user with one trained to behave realistically, and then mines the simulator's own hidden reasoning for a second, denser kind of feedback.