CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline

CLASHarness. Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline. By Yuxiang Lai and eight others, from Google, Emory University, and five other institutions. Published on the thirteenth of October. Its code repository is worked out: about three hundred and fifty source files, with a readme and a licence.

Vision-language models are increasingly placed inside multi-agent systems that automate complex computer-use workflows. Existing benchmarks for them mostly assume an environment that is consistent and cooperative. Real collaboration rarely works like that. Peer agents and sub-agents report states that are out of date. They issue instructions that contradict each other. They compete over shared permissions and files. An agent in that world has to check what others tell it and verify the ground truth in the environment itself.

Measuring that skill is hard for a practical reason. Writing conflict scenarios by hand cannot scale across complex systems. Letting a language model generate them freely brings its own failures: narrative drift, answers leaking into the task, and rubrics that cannot be satisfied.

CLASHarness, short for Conflict-Loaded Agent Scenario Harness, is a controllable framework for generating multi-agent conflicts in computer use that can be verified. It comes with a benchmark of six hundred multi-round tasks. What is being measured is conflict resolution: whether an agent notices that the information it was given disagrees with reality, and whether it can act correctly despite that.

Generation combines hierarchical scenario planning with an eight-step verified micro-pipeline. Two choices carry the reliability. The decisive visual facts are committed to before anything is rendered. And graders, a hybrid of code and semantic checks, are compiled directly from reference trajectories that were actually executed. Because of this, every task is guaranteed solvable by construction.

Fifteen frontier models are evaluated, and three core bottlenecks emerge. The first concerns whom to believe. When a sub-agent's output conflicts with the environment, weaker models follow the misleading information blindly, while stronger models inspect the visual evidence to verify ground truth.

The second is that resolving a conflict is much harder than finding one. Agents can spot inconsistencies, but when they must actively fix the system state, task success drops significantly. The third is coordination protocol, which remains a bottleneck for every model. Even frontier models fail on more than fifty-eight percent of closing actions, often forgetting to release a shared lock or to hand off a task.

The results describe a ladder. Noticing a conflict is the first rung, and stronger models climb it. Repairing the state is the second, and success falls there. Finishing cleanly, releasing what was held and passing on what was owed, is the rung where even frontier models mostly slip. The authors offer CLASHarness as a scalable, verifiable foundation for building multi-agent systems that hold up when their members disagree.