WideSWE: Can Coding Agents Coordinate Changes Across Repositories?
Baoyi Wang, Xingliang Wang, Jinyang Wu, Keming Wu, Chen Zhi, Jianwei Yin
Zhejiang University · Tsinghua University
Published September 27, 2026 · on the feed September 29, 2026 · ▲ 17 · arXiv 2609.33382
The abstract
Coding-agent evaluation has progressed from resolving individual issues to carrying out long-horizon development, yet task completion is still largely assessed within a single codebase. In software ecosystems, many features and bug fixes require coordinated changes across multiple repositories. We introduce WideSWE to evaluate coding agents on such cross-repository tasks. Mining and reviewing changes across 103 software ecosystems yields 120 real-world tasks, balanced between 60 bug fixes and 60 features. We derive prompts from related issues and pull requests. We systematically review and adapt hidden tests to support diverse correct implementations while preserving required behavior and regression checks. Across seven agent configurations, full task success ranges from 10.83% to 42.50%, with the configuration pairing Codex CLI with GPT-5.6-sol achieving the highest rate. Trajectories show agents failing to identify necessary changes, recognizing changes but leaving them unfinished, or modifying the required repositories without fully satisfying the request. To examine whether working on one repository at a time can alleviate these difficulties, we compare it with joint execution under identical prompts. Independent execution mainly recovers omitted work and is less effective at correcting previously attempted but unsuccessful implementations. Joint execution can use information from related repositories to guide implementation and verification. Code is available at https://github.com/ZJU-ACES-ISE/WideSWE.
The Brief
2:46A short podcast summary of the paper. Written and voiced by AI.
Transcript
WideSWE. Can Coding Agents Coordinate Changes Across Repositories?. By Baoyi Wang and five others, from Zhejiang University and Tsinghua University. Published on the twenty-seventh of September. Its code repository is worked out: about one thousand five hundred source files, with a readme.
Evaluation of coding agents has grown from resolving single issues to carrying out long-horizon development. One boundary has held throughout: completion is still judged largely inside a single codebase. Real software lives in ecosystems, and there many features and bug fixes need coordinated changes in several repositories at once. A fix that is correct in one place and missing in another is not a fix.
WideSWE measures exactly that: whether a coding agent can carry out a task that spans repositories, with success defined on the whole task rather than on any one repository in it. The unit of success widens from a codebase to the set of codebases a change has to touch. An agent that gets one repository right and leaves a dependent one untouched has not completed the task, however good its work on the first was.
Mining and reviewing changes across one hundred and three software ecosystems yields one hundred and twenty real-world tasks, split evenly between sixty bug fixes and sixty features. Prompts are derived from the related issues and pull requests. The hidden tests were systematically reviewed and adapted, so that they accept the range of correct implementations while still enforcing the required behaviour and checking for regressions. That adaptation matters for fairness. A test tied to one particular implementation would fail agents whose solutions differ but are still correct, so the tests were loosened in that direction and no further.
Across seven agent configurations, full task success ranges from ten point eight three percent to forty-two point five percent. The highest rate comes from pairing Codex CLI with GPT-5.6-sol. The trajectories reveal three ways of falling short. Agents fail to identify the changes needed. They recognise the changes but leave them unfinished. Or they modify the right repositories without fully satisfying the request.
A follow-up asks whether working one repository at a time eases these difficulties, compared against joint execution under identical prompts. Independent execution mainly recovers omitted work. It is less effective at correcting implementations that were attempted and failed. Joint execution, for its part, can draw on information from related repositories to guide both implementation and verification.
Even the best configuration completes well under half of the tasks. The two execution strategies divide the failures between them. Splitting the work up rescues what was forgotten, while keeping it together lets an agent use one repository to inform another. Neither, on the evidence given, covers everything the benchmark exposes. The code is public.
Paper Quest · 6 chapters · about 33 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Real fixes rarely stay in one repository. Benchmarks mostly do.
You evaluate coding agents. Benchmarks ask them to fix an issue or build a feature, and success is mostly judged inside one codebase.
The road ahead
- 1 · The problem ~7 min Coding-agent benchmarks mostly judge success inside one codebase, but real fixes and features often need matching changes in several repositories at once.
- 2 · The key idea ~9 min WideSWE counts a task solved only when every repository it touches passes, using real mined changes and fairly adjusted hidden tests.
- 3 · Choose your own research ~3 min
- 4 · How it works ~3 min
- 5 · The experiment ~6 min You’ll guess: On real cross-repository coding tasks, what full task success rate did the best of the evaluated coding-agent configurations reach?
- 6 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.