Allspark: Weak to Strong Transfer via Alternating Chain of Thought

Allspark. Weak to Strong Transfer via Alternating Chain of Thought. By Kaizhao Liang and four others, from the University of Texas at Austin, Together A I, and one other institution. Published on the twenty-sixth of September. No code repository is linked to it.

Progress in frontier models has revived interest in large-scale reinforcement learning. Yet generating rollouts from a large model is costly enough that even testing an RL recipe becomes expensive. The question here is whether that cost can be sidestepped: can reasoning improvements learned by a small, weak model benefit a larger, stronger one, without ever using the strong model's rollouts during training? If it can, the expensive part of the loop moves to a model that is cheap to run, and the strong model only has to show up at the end.

Allspark is a framework for training and inference built on alternating chains of thought. A weak teacher learns to reason in turns with a partner, and later that partner is swapped for a stronger one. The teacher's contribution is text inside a shared chain of thought, so what it learned can be carried to a different model.

During training, the weak teacher is paired with a frozen copy of the same model. The two alternate reasoning segments, and the frozen copy produces the final answer. At inference time a stronger student takes the frozen partner's place, and neither model is updated. Because the two communicate only through text, the teacher can steer students from other model families, including ones that use different tokenizers. Nothing about the arrangement requires the two models to share weights, vocabulary, or lineage. They only need to be able to read each other's segments of reasoning, and continue from them.

Two scales are studied. Controlled experiments with Qwen cover math and reasoning. Larger experiments with Inkling target ARC-AGI-2. The Inkling experiments show accuracy gains both within a model family and across families, including transfer to Kimi and to Nemotron. The benefits vary across inference settings, and the abstract says so directly. That caveat is part of the result: the transfer is shown to happen, but how much it helps depends on how the pair is run.

Train cheaply on the small side, then lend what was learned to something bigger through the only channel every model shares, which is text. The authors draw two directions from their findings: reusing one trained weak teacher across several strong students, and looking closely at the tradeoff between accuracy and the tokens spent to reach it.