τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions

τ-Emotion-Bench. Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions. By Rishitosh Kumar Singh and four others, from Arizona State University. Published on the fourteenth of October. Its code repository is worked out: about six hundred and fifty source files, with a readme and a licence.

In long-horizon tasks that involve calling tools, an LLM agent can fail for reasons beyond tool-use errors. It can also fail because it handles the user badly. Existing benchmarks largely assume users who cooperate and behave consistently. That leaves a question underexplored: what happens to an agent's reliability when the user's emotions vary?

τ-Emotion-Bench evaluates tool-calling agents across ten domains under user interactions that are conditioned on emotion in a controlled way. The emotional framing changes; the underlying task objective stays the same. That separation is what lets the benchmark attribute a change in agent success to how the user behaves rather than to what the user wants.

To scale both building the benchmark and training agents, the authors introduce Tracer, an agentic data-generation pipeline. It constructs tasks that are grounded and valid under policy, and collects trajectories through an interactive harness joining agent, user, and environment.

One feature of that harness does real work. Users who are dissatisfied can end an interaction that is going inefficiently. That gives an interaction-aware signal, which is used to filter out trajectories that lean too heavily on the user and to favour ones that use tools effectively.

Experiments run with two user simulators. Frontier agents remain unreliable: there are substantial gaps between their average task success and their consistent completion of tasks. Supervision generated by Tracer consistently improves Qwen3 models across scales. It generalizes to domains not seen in training, and it transfers to τ-Bench and to τ-Trait.

Behavioural analysis adds one more pattern. Higher task success is associated with a better balance between executing tools and following up with the user where that is necessary. Success, on this reading, is not only about calling the right tools, but about knowing when to ask.

The benchmark and the pipeline answer two halves of one question. τ-Emotion-Bench measures how fragile agents become when users stop being calm and cooperative. Tracer turns the same kind of interaction into training data, with an impatient user built in as a filter. Together the authors present them as a scalable framework for evaluating agent reliability across diverse user behaviours and for improving tool-calling agents through interaction-aware supervision.