Rufus-Air: An Open LLM Post-Training Recipe
Chia-Yuan Chang, Renyuan Cheng, Rui Feng, Xiaotian Han, Yuan He, Hongye Jin, Linwei Li, Shiyang Li and 14 others
Amazon
Published September 24, 2026 · on the feed September 25, 2026 · ▲ 22 · arXiv 2609.29421
The abstract
Rufus-Air is an open and reproducible post-training recipe on GLM-4.5-Air-Base (106B-A12B), organized as a serial pipeline of eight stages: SFT, Reasoning RL, Coding RL, Instruction-Following RL, General Agent, Coding Agent, Search Agent, and RLHF. We document the data, reward design, infrastructure, stage order, and stagewise results needed to reproduce the recipe. Stages progress from basic to advanced capabilities and from hard, verifiable rewards to softer judge-based signals. Training builds on open-source components and public data, much of it used as released, without new human annotation or an in-house distillation teacher. Our main findings are that (i) diverse, high-quality SFT establishes a strong capability floor; (ii) difficulty filtering keeps RL prompts within a productive learning range; (iii) reward reliability provides a practical principle for ordering stages; and (iv) infrastructure and engineering choices are part of the recipe, not just an implementation detail. Rufus-Air improves over the official GLM-4.5-Air post-trained release and is competitive with similarly sized open models.
The Brief
2:06A short podcast summary of the paper. Written and voiced by AI.
Transcript
Rufus-Air. An Open LLM Post-Training Recipe. By Chia-Yuan Chang and twenty-one others, from Amazon. Published on the twenty-fourth of September. No code repository is linked to it.
A post-training recipe is only reproducible if everything that went into it is written down: the data, the rewards, the infrastructure, and the order of the stages. This work sets out to provide exactly that, built from open-source components and public data, much of it used as released. There is no new human annotation and no in-house distillation teacher behind it.
Rufus-Air is that recipe, applied to GLM-4.5-Air-Base, a model of one hundred and six billion parameters with twelve billion active. Its organising idea is a serial pipeline whose stages move in two directions at once. They go from basic capabilities to advanced ones, and from hard, verifiable rewards to softer signals that come from a judge.
There are eight stages, in this order. Supervised fine-tuning first. Then reinforcement learning for reasoning, for coding, and for instruction following. Then three agent stages: a General Agent, a Coding Agent, and a Search Agent. Last comes RLHF. For each, the authors document the data, the reward design, the infrastructure, and the stagewise results, so the whole sequence can be rerun.
Four findings come out of it. Diverse, high-quality SFT establishes a strong capability floor. Difficulty filtering keeps reinforcement learning prompts inside a productive learning range. Reward reliability gives a practical principle for deciding the order of stages. And infrastructure and engineering choices are part of the recipe itself, not an implementation detail to be left out. On outcomes, Rufus-Air improves over the official GLM-4.5-Air post-trained release, and is competitive with open models of similar size.
The third finding explains the pipeline's shape. Put the stages whose rewards can be trusted first, and move toward judged, softer signals only once that floor is in place. The fourth explains why the paper is written the way it is: if engineering choices are part of the result, a recipe that leaves them out is not one anyone else can follow.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Eight stages, one model, and rewards of very different honesty. The first wall is deciding what runs when.
You are post-training an open base model in many stages: supervised examples, reinforcement learning for reasoning, coding and following instructions, three agent stages, and a final stage for open-ended quality.
The road ahead
- 1 · The problem ~6 min When post-training runs in many stages with rewards of different reliability, the order of the stages decides how much chance the model has to cheat.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Order the stages by how easily their reward can be gamed: hard, verifiable rewards first, judge-based rewards later, RLHF last.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min A serial pipeline: broad supervised fine-tuning, then verifiable-reward RL, then three agent stages, then RLHF on a learned reward.
- 6 · The experiment ~6 min You’ll guess: Built only from open components and public data, how did the recipe's model compare with the official GLM-4.5-Air post-trained release?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.