Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

Next-Concept Modeling. Fast, Parallel Coarse-to-Fine Text Generation. By Hermann Kumbong and eight others, from Stanford University and Together AI. Published on the ninth of October. Its code repository is public and real: about twenty source files.

Diffusion language models sample in parallel, which makes them faster at generation than autoregressive models, which write one token after another. That advantage matters more and more as inference becomes a dominant share of computational work. But diffusion language models have three weaknesses. Their quality falls off quickly when the number of sampling steps is cut, which is exactly where the speed would come from. They process the full sequence at every step. And they need many passes over the dataset before they reach good quality.

Next-Concept Modeling, or NCM, generates text coarse to fine, in two stages. First the text is encoded into a hierarchy of discrete codes. Then a model generates that hierarchy one level at a time: not one token at a time, and not the whole sequence at every step.

The encoder is a multiscale residual VQ-VAE, which maps text into the hierarchy of codes. A transformer then generates the levels in turn. Within each level, it predicts all the codes in parallel, so the number of sequential steps is set by the number of levels.

All comparisons are made at matched parameter scales. On LM1B, NCM achieves better generative perplexity than the strongest diffusion baseline while running twenty-one times faster. On OpenWebText, it is competitive with the strongest diffusion model, and ten times faster. On XSum, it matches that model's ROUGE-L score at thirty-one times the speed.

The authors also test reasoning, on Sudoku. There, NCM outperforms both autoregressive and diffusion baselines at every difficulty level, while using fewer function evaluations.

The speed comes from changing what gets generated in parallel. Instead of denoising a whole sequence step after step, the model fills in a short ladder of levels, each one all at once, from the coarsest to the finest.