DISCO: Distributed Long Context Scaling with Grounding-Reasoning Disaggregation
Guanzheng Chen, Viet Dac Lai, Subhojyoti Mukherjee, Branislav Kveton, Seunghyun Yoon, Franck Dernoncourt, Qizhe Xie, Trung Bui
National University of Singapore · Adobe Research
Published September 27, 2026 · on the feed September 29, 2026 · ▲ 3 · arXiv 2609.33485
The abstract
While Large Language Models (LLMs) advertise million-token context windows, reasoning quality often collapses as inputs grow -- a phenomenon termed context rot. This failure stems from a structural entanglement in monolithic architectures, where the massive search burden of contextual grounding exhausts the representational capacity needed for complex reasoning. To resolve this, we propose Grounding-Reasoning Disaggregation via DIStributed long COntext scaling (DISCO). Inspired by distributed computing frameworks like Apache Spark, DISCO partitions long context across a fleet of Worker LLMs dedicated exclusively to parallel, localized grounding. A central Driver LLM, trained via Reinforcement Learning (GRPO) to optimize planning, orchestrates execution by dynamically mapping queries into atomic extraction tasks and reducing the gathered evidence to synthesize a final answer. By isolating reasoning from raw context noise, DISCO effectively eliminates context rot. On RULER-QA (1M tokens), it maintains 78.4% accuracy where standard baselines collapse. Furthermore, it outperforms full-context models by up to 9.8 points on LongBench v2 and matches frontier models like Gemini-3-Pro-Preview while reducing inference costs by over 80%, establishing a highly efficient paradigm for robust long-context inference.
The Brief
2:14A short podcast summary of the paper. Written and voiced by AI.
Transcript
DISCO. Distributed Long Context Scaling with Grounding-Reasoning Disaggregation. By Guanzheng Chen and seven others, from the National University of Singapore and Adobe Research. Published on the twenty-seventh of September. No code repository is linked to it.
Large language models now advertise context windows of a million tokens, yet reasoning quality often collapses as the input grows. The phenomenon has a name, context rot. The authors locate its cause in how a single monolithic model is asked to do two jobs at once. One job is grounding: searching the context for what is relevant. The other is reasoning over what was found. With enough input, the sheer search burden of grounding exhausts the representational capacity that complex reasoning needs. The two are structurally entangled, and the reasoning loses.
DISCO, which stands for distributed long context scaling, pulls the two jobs apart. Its inspiration is distributed computing frameworks such as Apache Spark. Instead of one model holding everything, the long context is partitioned across a fleet of Worker models whose only task is grounding, done in parallel and locally on each piece.
Above them sits a central Driver model. The Driver plans. It maps an incoming query into atomic extraction tasks, sends those out to the workers, then reduces the evidence they gather into a final answer. The Driver is trained with reinforcement learning, specifically GRPO, to optimize that planning. The effect is that the model doing the reasoning never touches the raw, noisy context. It sees only the evidence that comes back.
The authors report that this isolation effectively eliminates context rot. On RULER-QA at one million tokens, DISCO keeps seventy-eight point four percent accuracy, where standard baselines collapse. On LongBench v2 it outperforms full-context models by as much as nine point eight points. And it matches frontier models such as Gemini-3-Pro-Preview while cutting inference costs by more than eighty percent.
The borrowed analogy carries the whole design. Map the work out to many hands, then reduce what they bring back, so that no single participant has to hold everything. DISCO applies that division of labour to a long document, and the claim is that splitting search from thought fixes accuracy and cost together rather than trading one for the other.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A million tokens fit in the window. The reasoning does not survive them.
Your model advertises a million-token context, but as the input grows its reasoning collapses. Answers that are easy on a short document go wrong on a long one.
The road ahead
- 1 · The problem ~7 min Long inputs break reasoning because one model must both search a huge context and reason over what it finds, and the search crowds out the reasoning.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min DISCO separates grounding from reasoning: Worker models extract evidence from pieces of the context, and a Driver model reasons only over that evidence.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min DISCO partitions the context across Workers, maps the question into atomic tasks, extracts evidence on each shard in parallel, then reduces it into an answer.
- 6 · The experiment ~6 min You’ll guess: While matching frontier models like Gemini-3-Pro-Preview, by how much did DISCO's Driver-and-Workers setup cut inference costs?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.