MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
Core Team, Zongming Qiao, Ziyue Hua, Zirui Ou, Zihao Yue, Zihan Jiang, Zhuo Huang, Zhiyang Chen and 141 others
LLM-Core Xiaomi
Published October 8, 2026 · on the feed October 9, 2026 · ▲ 77 · arXiv 2610.11959
The abstract
Reinforcement learning (RL) is the central training paradigm for advancing large foundation models towards self-improvement. This report introduces the MiMo-V2.6 series, an omni-modal family that pushes the frontier of model intelligence by scaling RL compute. Prior to RL, we conduct mid-training on a broad multimodal corpus to provide ample exploration space, and build a solid infrastructure on the pretrained hybrid-SWA architecture to support subsequent scale-up. We scale RL compute along three dimensions: (1) larger batches and higher throughput, with an asynchronous training that consumes 1,568 samples and 2.7-3.7B tokens per step at context lengths of up to 1M; (2) more diverse and complex environments, spanning code, general, visual, and cyber domains under a mixture of agent harnesses; and (3) more grader compute, via groupwise agentic grading that yields more accurate reward signals for long-horizon tasks and steers the model towards shorter, more token-efficient solutions. To keep training stable at scale, we freeze the MoE router and establish a multi-layer defense against reward hacking. We further build infrastructure for mixed-task agentic RL, including a unified trajectory representation, high-concurrency multi-framework rollout, decoupled control and data planes, and training-inference consistency. We open-source the training dynamics, RL environments, and RL framework to facilitate reproduction and further research on scaled RL and model self-improvement.
The Brief
2:38A short podcast summary of the paper. Written and voiced by AI.
Transcript
MiMo-V2.6. Scaling Reinforcement Learning Towards Self-Improvement. By Core Team and one hundred and forty-eight others, from the LLM-Core team at Xiaomi. Published on the eighth of October.
The report opens from a position: reinforcement learning is the central training paradigm for pushing large foundation models toward self-improvement. If that is true, the question becomes how far the reinforcement learning compute can be scaled, and what has to be built to keep it working at that scale. Scaling here is not only about spending more. It touches the data the model sees beforehand, the systems that run training, the environments it learns in, and the way its outputs are judged.
MiMo-V2.6 is an omni-modal model family that pushes model intelligence by scaling reinforcement learning compute. Before that stage, the team runs mid-training on a broad multimodal corpus, to give the model ample exploration space, and builds infrastructure on the pretrained hybrid-SWA architecture to support the scale-up that follows.
Compute is scaled along three dimensions. The first is larger batches and higher throughput. Training is asynchronous and consumes one thousand five hundred and sixty-eight samples per step. Each step also takes in between two point seven and three point seven billion tokens, at context lengths up to one million.
The second dimension is more diverse and more complex environments, covering code, general, visual and cyber domains under a mixture of agent harnesses. The third is more compute spent on grading. Groupwise agentic grading yields more accurate reward signals for long-horizon tasks, and steers the model toward shorter solutions that use fewer tokens. Better grading matters most where tasks are long, because a weak reward signal over a long trajectory gives the model little to learn from.
Stability at this scale needs its own measures. The team freezes the MoE router and sets up a multi-layer defense against reward hacking. For mixed-task agentic training they build a unified trajectory representation, high-concurrency rollout across multiple frameworks, separate control and data planes, and consistency between training and inference.
The abstract does not report benchmark scores. What it offers instead is openness. The team releases the training dynamics, the reinforcement learning environments, and the reinforcement learning framework, so that others can reproduce the work and study scaled reinforcement learning and model self-improvement for themselves.
This is a report about engineering for scale more than about a single new algorithm. Bigger batches, harder environments, more careful grading, and a set of stabilizers and infrastructure that keep all of it from falling apart. The release of the dynamics and environments is what turns it from a description into something other groups can build on.
Paper Quest · 7 chapters · about 38 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every quest starts with a wall. Here it is how far reinforcement learning can be pushed.
You believe reinforcement learning is the way to push a large model toward improving itself. The question is how far you can scale it, and what breaks when you try.
The road ahead
- 1 · The problem ~7 min Reinforcement learning could push a large model to improve itself, but scaling it up breaks in several places at once, not just in the step count.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Prepare the model to explore, then scale RL compute along batches, environments and grading together, with measures that keep the huge run stable.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Mid-train for exploration, run huge asynchronous RL batches over diverse environments, and grade passing solutions by quality, with offline rubrics or online group comparison.
- 6 · What they found ~3 min
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.