Large Flow Language Models

Large Flow Language Models. By Arnav Chavan and six others, from UC Berkeley, Amazon, and two other institutions. Published on the twentieth of October. Its code repository is worked out: about forty source files, with a readme and a licence.

Continuous flow language models offer a route to generating text in parallel, and they come with tools for control and acceleration that discrete diffusion does not have. Parallel here means producing many tokens at once, rather than strictly one after another. What has been less established is how to build such models effectively at the scale of billions of parameters. The promise is there; the recipe for large models is not.

Lafa, short for Large flow language model, is presented as a fast, high-performing flow language model, scaled to eight billion parameters in a data-efficient way. Its central design is blockwise. Over the prefix, attention is causal, the way an autoregressive model reads text. Within each block, the flow is bidirectional and asynchronous.

That split is what makes both training and inference efficient. On the training side, Lafa can be built by adapting pretrained autoregressive models rather than starting from nothing. On the inference side, the causal prefix allows prefix caching, and the blocks allow parallel decoding, block by block.

The flow brings a further benefit. Flow training preserves, and can even improve, the capabilities of the pretrained autoregressive model, through what the authors describe as a multi-token prediction effect. They put that effect to use in a self-speculative decoding scheme: within a single model, flow-based drafts are proposed and then verified autoregressively. The same network plays both parts, drafter and verifier, so no second model is needed to speed up the first.

Lafa is trained at three scales, half a billion, seven billion, and eight billion parameters, each with fewer than thirty billion tokens, which the authors report as more efficient than previous diffusion models. On instruction following, reasoning and coding, the learned flow matches or exceeds the quality of autoregressive generation. It can also be distilled into flow maps for generation in one step or a few steps.

The self-speculative scheme delivers two point five four times the throughput of an autoregressive model at comparable quality, and it outperforms both multi-token prediction and discrete diffusion methods.

Rather than discarding an autoregressive model to try a new paradigm, Lafa grows a flow on top of one, then lets the autoregressive side check what the flow drafts. The authors take their findings to show that continuous flows can serve as a scalable foundation for language modelling.