StartLux-Decision: AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions

StartLux-Decision. AutoThink Decision Models with Reinforcement Learning for Calibrated Deliberative Decisions. By Xialie Zhuang and five others, from StartLux, the University of Hong Kong, and one other institution. Published on the twelfth of October. Its code repository is worked out: about ninety source files, with a readme and a licence.

A decision model faces a trade between two virtues. Answering in a single pass is fast and cheap. Reasoning explicitly can help when a case is hard, but spending that effort on every case is wasteful, and long reasoning traces can run past the generation limit without ever finishing. The question this work takes on is how to get the speed of direct prediction on easy decisions and the benefit of deliberation on uncertain ones, inside one family of models.

StartLux-Decision is that family, spanning zero point eight billion to twenty-seven billion parameters, and it comes in two variants. The Instant models keep single-pass inference, but use bidirectional attention in their full-attention layers to improve decision accuracy. The AutoThink models are adaptive and confidence-guided: they answer confident cases directly and invoke explicit reasoning only for decisions they are uncertain about.

Training runs in three stages. The first establishes decision capabilities the two variants share. The second specializes them. The third equips AutoThink with reasoning, through reasoning supervision followed by reinforcement learning.

That reinforcement learning step has its own name: Reinforcement Learning for Calibrated Deliberative Decisions, written RLCD squared. It uses a frozen model and proper scoring rules to reward improvements in the quality of decision probabilities, not merely whether the answer was right. On top of that, it applies a soft penalty to reasoning traces that run overlong.

The effect of RLCD squared on length is large. It raises the share of reasoning traces that terminate within the generation limit by up to forty-eight percentage points, and cuts average reasoning length by up to sixty-five percent, while largely preserving decision accuracy.

On the public subset of Decision Index zero point three, the Instant models outperform the strongest evaluated open-weight decision models of comparable scale by seven point four to sixteen point nine points. The largest AutoThink model, at twenty-seven billion parameters, reaches seventy-three point four. The authors also introduce a benchmark called DecisionPivot: computational decisions that can be verified programmatically, built from contrast pairs that differ in a single detail which changes the answer. There, AutoThink beats its Instant counterparts by eleven point five to eighteen point four points in pair accuracy, and leads among the systems evaluated.

Pairs that hinge on one detail are where deliberation should pay, and that is where AutoThink's margin over Instant shows up. The release is built to be used: the models run on vLLM, llama.cpp and MLX, with quantized variants for lower memory use. The through line is a model that decides when to think, and a reward that cares about how well calibrated its probabilities are, and about whether its reasoning finishes.