Miles v0.1: Production-Level Post-Training
RadixArk, Tom Chen, Mao Cheng, Shi Dong, Kangrui Du, Yanbin Jiang, Jiajun Li, Yiming Li and 5 others
RadixArk
Published September 8, 2026 · on the feed September 9, 2026 · ▲ 58 · arXiv 2609.08368
The abstract
We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each stage of the reinforcement-learning (RL) training loop around a single principle: components should be verified, clean, and customizable. With accuracy, efficiency, reliability, and scalability as first-class goals, Miles aims to make frontier-scale RL accessible to researchers and enterprises alike. This report walks through the system end to end: rollout engines built on SGLang, a trainer with a choice of two backends (NVIDIA Megatron-LM and PyTorch FSDP), and three weight-synchronization transports for different deployment topologies. Beyond full-parameter RL, Miles also supports LoRA RL, on-policy distillation, supervised fine-tuning, and true-on-policy rollout-training alignment, and extends the same architecture to diffusion models. We close with an end-to-end case study: fully asynchronous agentic RL on a GLM-5.2 744B-A40B model over terminal-use coding tasks, running on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. Miles is open-sourced at https://github.com/radixark/miles, with the project website at https://miles.radixark.com.
The Brief
2:25A short podcast summary of the paper. Written and voiced by AI.
Transcript
Miles v0.1. Production-Level Post-Training. By RadixArk and twelve others, from RadixArk. Published on the eighth of September. Its code repository is worked out: about one thousand five hundred source files, with a readme and a licence.
Miles v0.1 is a full-stack, production-ready system for frontier post- training, and it is open-sourced. It builds on the clean design of slime, and it aims to make reinforcement learning at frontier scale accessible to researchers and enterprises alike.
Every stage of the reinforcement-learning training loop is designed around one principle: components should be verified, clean, and customisable. Accuracy, efficiency, reliability, and scalability are all treated as first-class goals. The report walks through the system end to end. Rollout engines are built on SGLang. The trainer offers a choice of two backends, NVIDIA Megatron-LM and PyTorch FSDP. And there are three weight-synchronisation transports, each suited to a different deployment topology.
Beyond full-parameter reinforcement learning, Miles supports LoRA RL, on- policy distillation, supervised fine-tuning, and true-on-policy alignment between rollout and training. The same architecture extends to diffusion models. So a single system covers not only the main reinforcement-learning loop but the neighbouring forms of post-training that usually live in separate tools, and it reaches beyond language models as well.
The report closes with an end-to-end case study. It runs fully asynchronous agentic reinforcement learning on a GLM-5.2 model, the 744B-A40B configuration, over coding tasks that involve using a terminal. The run uses sixty-four NVIDIA GB300 GPUs, with a median step time of two hundred and sixty-three seconds over the first thirty measured steps. It puts the system's stated goals of efficiency, reliability and scalability to work on an asynchronous, agentic workload.
It is meant for researchers and enterprises that want to run reinforcement learning for post-training at frontier scale, and for those who need to adapt the pipeline, since customisable components are one of its stated principles. The choice of backends and synchronisation transports lets it fit different hardware arrangements.
The case study shows the system carrying a seven hundred and forty-four billion parameter model through agentic training on real hardware, and the long list of supported modes shows how much of post-training one codebase is meant to cover. The code and a project website are both public.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Post-training at frontier scale is a juggling act. Here is what keeps dropping.
You run reinforcement learning to post-train a huge mixture-of-experts model. Rollouts now span many turns, call tools, and act in outside environments.
The road ahead
- 1 · The problem ~6 min Post-training a huge model with reinforcement learning now wastes hardware on waiting, and tiny numerical gaps can quietly spoil every update.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Run the rollout, train and weight-update loop asynchronously, record trajectories token-exact, and build it from verified, replaceable parts.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min SGLang engines generate trajectories, the trainer updates the policy, and a weight transport sends the new weights back to the engines.
- 6 · The experiment ~6 min You’ll guess: In the case study of fully asynchronous agentic RL on GLM-5.2 across 64 GB300 GPUs, how long did a typical training step take?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.