LEV: Thinking Fast and Deep with Looped Decision Models

LEV. Thinking Fast and Deep with Looped Decision Models. By Dingjie Song and three others, from Lehigh University, the University of British Columbia, and one other institution. Published on the fourteenth of October. Its code repository is worked out: about eighty source files, with a readme and a licence.

A decision model answers a typed question about an input by giving one probability per option. It does this in a single forward pass, without generating any text. In dual-process terms it is a fast System 1: it answers at once. But its depth is fixed when it is trained, so an easy question and a hard one get exactly the same amount of computation. The alternative, models that reason in generated text, can think for longer, but every extra step of thought is paid for in decoded tokens.

LEV takes a third route. It is a decision model that thinks deeper by running its layers again, in a loop, instead of producing tokens. More depth means more passes through the same layers, not more text.

Three choices make that work. LEV starts from a language model that was pretrained as a loop. It reads the options after every loop step, not only at the end. And it trains all the steps jointly. The consequence is that a single checkpoint can answer at any depth you choose.

Depth can also be chosen per question. A small stopper decides when to stop looping, using a threshold fitted on unlabelled requests, so no labels are needed to set it.

On three looped backbones, accuracy rises with depth while the weights stay the same, and most of the rise is on hard questions. The clearest case is a two point six billion parameter LEV on JevBench, which goes from sixty-three point two percent at depth one to seventy-six point two percent at depth four.

Letting the stopper set depth on demand saves between fifteen and twenty-four percent of the computation, for a cost of less than one accuracy point. That is level with, or slightly better than, a fixed mix of depths. On a locked test set, the one point four billion and three billion parameter models beat both dense, non-looped models and cascades of the same latency, by five point four to eight point eight points, with latency held below fifty milliseconds.

Here, the dial between fast and deep is a loop count rather than a choice of model. The same weights answer quickly when one pass is enough and keep going when a question is hard, and none of that extra thinking has to be written out as text.

LEV. Thinking Fast and Deep with Looped Decision Models. By Dingjie Song and three others, from Lehigh University, the University of British Columbia, and one other institution. Published on the fourteenth of October. This is the long version: the whole paper rather than the abstract.

A decision model answers questions that come with a fixed menu of options. It reads the input once and returns one probability per option, in tens of milliseconds, without writing any text. The authors call it a System 1. Its weakness is that its depth is fixed at training time, so an easy question gets exactly as much computation as a hard one. The alternative is a generative model that reasons in text first. On JevBench that raises hard-tier accuracy by nine to thirteen points, but makes the median hard item three to four times slower, and every question pays for its reasoning, easy ones included. LEV is an attempt at something in between: a decision model that thinks longer only where it needs to, by rerunning its own layers instead of generating tokens.

Two findings shape the design. First, extra computation pays off on few questions. On the two hundred and thirty-one public JevBench items, the four billion parameter Kev gets twenty-eight more right than its sub-billion sibling, all from the standard and hard tiers, because both already answer every easy item. The small model's own confidence finds the questions that matter: sending its least confident half to the large model nearly matches the large model on everything.

Second, a loop bolted onto an ordinary pretrained model stays shut. The authors looped eight middle layers of a small Qwen backbone behind gates that start at zero. After training, the gates were still near zero, and accuracy moved by less than half a point. A closed loop gets little gradient to open, and an open one pushes hidden states away from what later layers expect.

A request holds a shared state followed by questions, each listing its options and ending in a decide token. A block-causal mask lets each question see only the state and itself, so the state is encoded once. A pointer head scores each option by comparing its hidden state with the one at the decide token.

LEV swaps in a natively looped backbone. Two Ouro models run their layer stack four times and were pretrained with a loss after every step. A Nanbeige model runs its stack twice and was supervised only on its last step. One shared pointer head reads the options after every step, and training averages the loss over all steps, so every depth is a working decision model. Stopping at depth two of four costs about half the full pass, and a further step resumes from the previous hidden states without rereading the input.

The stopper sets each question's depth. After each step, a small logistic regression reads features of the current and previous readouts, such as the top probability, the margin, and whether the answer just changed. The crucial choice is what it predicts: not whether the answer is correct, but whether it already agrees with the deepest step's answer. Its threshold is the cheapest one under which ninety-eight percent of final answers match the deepest ones. Neither needs a single label, so a deployment can fit the stopper on its own unlabelled traffic.

On the same weights, accuracy rises with depth on every backbone and panel. The two point six billion LEV goes from sixty-three point two percent at depth one to seventy-six point two at depth four on JevBench, with most of the gain in the first extra step. The gain lands on hard questions: from depth one to four, it nets nineteen of the hundred and eleven hard items while the easy tier is nearly solved throughout. The stopper follows difficulty without seeing it, spending one point two steps on easy items and three on hard ones.

The comparison that tests the thesis is against the fixed-depth frontier: serving a share of requests at one depth and the rest at the next. On the held-out panels, on-demand depth stays within one point of the deepest readout at seventy-six to eighty-five percent of its cost. Against the frontier, though, the two point six billion model gains only a fifth of a point, and the three billion model one point. A locked test, read once, repeats the picture. The ablations locate the gain. A single top-probability threshold set the same label-free way comes within a fifth of a point of the full stopper, while the usual confidence threshold fitted on labelled data falls below the frontier for both Ouro models.

The paper has a limitations section. Public looped backbones only span one point four to three billion parameters, so how far the depth curve extends is open. The JevBench results rest on its public items. Savings are counted in loop steps, and turning them into wall-clock time needs serving that batches requests at different depths. Latency is measured one request at a time, without optimised kernels.

The results add two more. The latency advantage is not a computation advantage: counted in floating-point operations, every LEV model is at or below the dense options, and part of its low-latency win comes from a gap where no dense option answers between twenty-two and seventy-nine milliseconds. And the stopper's target has a ceiling. For twelve point six percent of LevBench questions a shallower depth is right while the deepest is wrong, and predicting agreement with the deepest answer can never recover them.

The idea to carry away is that a natively looped backbone turns depth into a setting chosen at serving time. One checkpoint becomes a family of operating points, and a stopper that needs no labels picks among them per question rather than per deployment. Its win over a fixed mix of depths is small. The larger result is the depth curve itself: gains concentrated on hard questions, from weights that never had to learn to loop during decision training.