QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents
Weiqi Wang, Yuxin Zhou, Mouxiang Chen, Siyuan Zhang, Yi Zhang, Yuyan Luo, Zhiyu Yin, Chencan Wu and 4 others
Alibaba Group · University of Science and Technology of China · Tsinghua University
Published September 27, 2026 · on the feed September 29, 2026 · ▲ 41 · arXiv 2609.33848
The abstract
Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of model--environment interactions, and nearly 1M tokens per rollout. Applying online reinforcement learning (RL) to such executions poses two fundamental challenges: (1) severe execution variance and prolonged rollout delays cause massive GPU idling; and (2) complex non-linear branching generates massive trajectory redundancy, crippling training efficiency. To address these, we presents QwenGyre, an end-to-end framework for xlong-horizon online RL. QwenGyre elastically reallocates GPUs between rollout and training without interrupting live executions, while its trajectory processor reconstructs branching histories, scores partial progress, and deduplicates redundant paths to bound training costs. Scaled to our flagship model, Qwen~3.8 2.4T, with 700K tokens per rollout, QwenGyre yields a 6.0% absolute gain on NL2RepoBench (52.5% to 58.5%) in 48 steps. Across our evaluations on diverse domains of training datasets, QwenGyre delivers up to 1.85times and 1.78times speedups over Colocate and Async, respectively.
The Brief
1:59A short podcast summary of the paper. Written and voiced by AI.
Transcript
QwenGyre. An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents. By Weiqi Wang and eleven others, from Alibaba Group, the University of Science and Technology of China, and one other institution. Published on the twenty-seventh of September. No code repository is linked to it.
Language model agents are taking on tasks with extremely long horizons. A single run can last hours, involve hundreds of exchanges between the model and its environment, and reach nearly one million tokens per rollout. Online reinforcement learning on runs like that faces two basic problems. Execution times vary wildly and rollouts drag on, so large numbers of GPUs sit idle. And the runs branch in complex, non-linear ways, which piles up redundant trajectories and cripples training efficiency.
QwenGyre is an end-to-end framework for online reinforcement learning at this scale, and it answers each problem with its own mechanism. For idle hardware, it is elastic: it moves GPUs between rollout and training without interrupting executions that are still live.
For redundancy, it has a trajectory processor. The processor reconstructs the branching histories of a run, scores the partial progress along them, and removes duplicate paths, which keeps the cost of training bounded. Rather than training on every branch of a sprawling run as if each were new, the framework trains on what is actually distinct.
The authors scale it to their flagship model, the one the abstract calls Qwen 3.8 two point four T, with seven hundred thousand tokens per rollout. On NL2RepoBench, QwenGyre gives an absolute gain of six points, from fifty-two point five percent to fifty-eight point five percent, in forty-eight steps. Across training datasets from diverse domains, it runs up to one point eight five times faster than a Colocate setup, and up to one point seven eight times faster than an Async one.
In short, two sources of waste, two remedies. Hardware that would have waited on a slow run is lent back to training, and trajectories that repeat each other are thinned out before they reach the training bill.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Hours-long agent runs, a million tokens each, and GPUs waiting on the slowest. Start there.
You are training a coding agent with online reinforcement learning. A single run can last hours, take hundreds of exchanges with its environment, and reach nearly a million tokens.
The road ahead
- 1 · The problem ~7 min Very long agent runs leave GPUs idle while slow runs finish, and their branching records repeat each other, wasting training compute.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min QwenGyre moves GPUs between rollout and training as work changes, and turns each branching run into a scored, deduplicated tree of paths.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Rebuild each run as a tree, score its progress, cut it to a few distinct paths, then train on those.
- 6 · The experiment ~6 min You’ll guess: Compared with a colocated setup that alternates the whole GPU pool between rollout and training, how much faster did QwenGyre train at best?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.