One Model to Loop Them All: Elastic Latent Recurrence for Language Models

One Model to Loop Them All. Elastic Latent Recurrence for Language Models. By Faisal Ladhak and five others, from NVIDIA. Published on the eighth of October. No code repository is linked to it.

A latent recurrent language model runs a shared block of layers over and over. That makes one checkpoint flexible: change the loop count, and you trade inference compute for accuracy. The catch is the single loop. Run once, such models usually fall well short of a non-recurrent model of the same size. Call it a one-loop tax. Whenever latency matters, that tax makes a separate non-recurrent model the better choice, and the flexibility goes unused. A model that can stretch to more compute is not much use if its cheapest setting is the one nobody would deploy.

Elastic Latent Recurrence, or ELR, is a training recipe for looped language models that removes the tax. At one loop, an ELR model costs the same to serve as a non-recurrent model of the same size, and when trained on the same data it reaches similar accuracy. Every loop added after that buys more accuracy in exchange for more inference compute. The bottom rung is level with the plain model, and the rungs above it are extra.

Three choices make it work. A gated state update with a learned step size. Loop counts that are mixed within each optimizer update. And conditioning each pass on its loop index and on the budget. The first two protect accuracy at one loop. The third lowers the loss at every additional loop. The division of labour is clean: two choices remove the tax, and one makes the extra loops pay.

Several refinements keep the costs down. Mixed depths can be brought in late in pretraining, which limits the added training cost. With zero-initialized conditioning added at the switch, this late schedule actually reaches lower loss than mixed-depth training from scratch. In mixture-of-experts models, reusing the first loop's expert routes on later loops cuts mixed-depth update time by eleven point seven percent, with no measurable change in accuracy. And at inference, keeping only the last loop's attention KV cache holds attention KV memory at its one-loop level, while keeping most of what the extra loops gain.

The recipe is tested on two hybrid models: a dense model at four billion, and a 30B-A3B mixture-of-experts model. At one loop, ELR sits zero point five two points below its matched baseline on the dense model, and one point zero four points above it on the mixture-of-experts model, on a ten-task average. Four loops then add seven point two five points to the dense model, and four point seven five points to the other.

Carried through post-training, the recipe produces Koi-4B, a single checkpoint. At one loop it scores an estimated eighteen on the Artificial Analysis Intelligence Index, matching Qwen3.5-4B. At four loops it scores twenty-nine. That is the same set of weights in both cases, with nothing changed but how many times the shared block runs.

The promise of looping was always one model for many budgets. What kept it from being used was the cheapest setting, where the looped model lost to a plain one. With that setting brought level, the same weights can serve the latency-bound case and, a few loops later, the accuracy-bound one.