CrossSWE: Benchmarking Cross-Repository Joint Development for Coding Agents
Silin Chen, Yuling Shi, Heng Lian, Zihan Zhang, Xiaodong Gu, Haotian Lin, Zhensu Sun
Shanghai Jiao Tong University · Xidian University · Singapore Management University
Published November 1, 2026 · on the feed November 3, 2026 · ▲ 25 · arXiv 2611.01484
The abstract
As coding agents achieve increasingly high success rates on software engineering benchmarks, recent efforts have sought to increase task complexity through longer horizons and more extensive code modifications. However, real-world software development often spans multiple interdependent repositories, introducing additional challenges in dependency reasoning, change propagation, and interface consistency. Rather than further increasing complexity within individual repositories, we introduce CrossSWE, a benchmark for cross-repository joint development comprising 171 real-world instances spanning 165 repositories, 20 technical domains, and 13 programming languages. CrossSWE verifies the necessity of coordinated changes by testing whether partial repository modifications are insufficient to achieve the intended behavior. Our evaluation of five LLMs reveals substantial challenges, with the strongest model, GPT-6.1 Sol, achieving only 58.48% Pass@1. Furthermore, cross-repository inconsistencies emerge as the most prevalent failure mode (44.2%), highlighting the difficulty of maintaining consistency across interdependent repositories.
The Brief
1:55A short podcast summary of the paper. Written and voiced by AI.
Transcript
CrossSWE. Benchmarking Cross-Repository Joint Development for Coding Agents. By Silin Chen and six others, from Shanghai Jiao Tong University, Xidian University, and one other institution. Published on the first of November.
Coding agents now reach increasingly high success rates on software engineering benchmarks. Recent efforts to raise the difficulty have done it from inside a single repository: longer horizons, more extensive code modifications. Real-world software development often does not stay inside one repository. It spans several interdependent ones, and that brings challenges of its own: reasoning about dependencies, propagating a change from one place to another, and keeping interfaces consistent where the pieces meet. Making a single repository harder does not test any of that.
CrossSWE is a benchmark for cross-repository joint development. Rather than adding complexity within one codebase, it asks an agent to make coordinated changes across several. It comprises one hundred and seventy-one real-world instances. They span one hundred and sixty-five repositories, twenty technical domains, and thirteen programming languages.
The central design check is necessity. CrossSWE verifies that the coordinated changes are actually required, by testing whether modifying only part of the repositories is insufficient to achieve the intended behavior. An instance that could be solved by touching one repository alone would not be measuring joint development, and this test is what rules that out.
Five LLMs were evaluated, and the results show substantial challenges. The strongest model, GPT-6.1 Sol, reaches only fifty-eight point four eight percent Pass@1. Looking at how attempts fail, cross-repository inconsistencies are the most prevalent failure mode, at forty-four point two percent.
Those two numbers point the same way. Even the best model misses roughly two instances in every five, and the commonest reason is not a bug in one place but a mismatch between places. Keeping interdependent repositories consistent with each other is where the difficulty concentrates.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free