The Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models

The Missing Primitive. Diagnosing and Repairing Mathematical Reasoning in Large Language Models. By Shuo Xing and nine others, from Texas A and M University, Harvard University, and four other institutions. Published on the first of October. Its code repository is public and real: eighteen source files.

Large language models now show striking ability on frontier mathematical problems. Whether they hold the structural mathematical understanding beneath their solutions is less clear. A correct final answer says that a problem was solved. It does not say which parts of the reasoning the model actually commands, and which it got past some other way. The authors present this as a first step toward studying that understanding systematically, from diagnosing its separate capabilities to using what the diagnosis shows to improve post-training.

The probe is a notion the authors call a Mathematical Primitive, aimed at structural mathematical understanding. Around it they propose a new benchmark that evaluates mathematical reasoning along four distinct dimensions: Discovery, Generation, Digestion and Execution.

The design point is separation. Rather than one score for a solved problem, reasoning is evaluated along each of the four dimensions, so that a model's result can read as a profile instead of a single number. Two models with the same accuracy might then turn out to differ sharply, one weak at finding what to use and the other weak at carrying it through. That separation is also what makes a targeted repair possible later on.

The systematic diagnosis finds three things. Solution accuracy masks distinct capability profiles. Primitives unlock substantial latent capacity for execution. And Discovery is the dominant bottleneck in mathematical reasoning. A further analysis of post-training adds that failures limited by discovery are particularly amenable to repair.

Building on that, the authors introduce a self-distillation framework in which primitives are privileged. It selectively transfers reasoning guided by primitives into the student model. Extensive experiments show that it consistently improves mathematical reasoning over baselines, across model scales and on challenging benchmarks.

The diagnosis and the repair line up. If the step models most often miss is discovering the right primitive, then transferring primitive-guided reasoning into a student aims at exactly that step, and the experiments report that it helps.