Thought without systematicity? Evaluating reasoning models on rule induction tasks
Simon Schug, Brenden M. Lake
Princeton University
Published September 12, 2026 · on the feed September 15, 2026 · ▲ 3 · arXiv 2609.13948
The abstract
A central tenet of human cognition is systematicity, the principle that understanding one concept is inherently tied to understanding close variations of that concept. Do reasoning models robustly exhibit such systematicity? If so, we would expect consistent performance on structurally equivalent variants of the same task. Here, we extend established rule induction tasks from cognitive science to assess the systematicity of thought in current reasoning models. Each task family has compositional structure that we use to create structurally equivalent task variations through task isomorphisms such as recombination and substitution. We find that despite being able to correctly solve a task, models often fail on structurally equivalent variants of the same task. These findings suggest that many model behaviors lack systematicity, rendering it difficult to robustly establish the cognitive abilities of reasoning models beyond the particular contexts they were evaluated in.
The Brief
1:48A short podcast summary of the paper. Written and voiced by AI.
Transcript
Thought without systematicity? Evaluating reasoning models on rule induction tasks. By Simon Schug and one other, from Princeton University. Published on the twelfth of September. Its code repository is public and real: about twenty-five source files.
Systematicity is a central tenet of human cognition: the principle that understanding one concept is inherently tied to understanding close variations of it. The authors ask whether reasoning models robustly exhibit it. If they did, their performance should be consistent across structurally equivalent variants of the same task. The finding is that it is not. Models are often able to solve a task correctly, and then fail on structurally equivalent variants of that same task.
The test extends established rule induction tasks from cognitive science to current reasoning models. Each task family has a compositional structure, and the authors use that structure to create variations that are structurally equivalent to the original, through task isomorphisms such as recombination and substitution. A variant changes the parts or how they are combined, while the underlying structure stays the same, so a model that had grasped the structure should handle the variants alike. Whether the parts are recombined or substituted, the logic that solves the original should carry over to the variant.
The claim is that many model behaviors lack systematicity, not all of them, and it is established on rule induction tasks. What it bears on is how far an evaluation can reach. A model's success in one context becomes weak evidence of a cognitive ability outside that context, which makes it difficult to robustly establish the abilities of reasoning models beyond the particular settings in which they were tested.
A correct answer, on this view, is a single point. Establishing an ability means checking the neighbouring points as well, and for current reasoning models the authors find that those neighbours often come out wrong.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model passes a test. The question is what that pass actually proves.
A reasoning model solves a puzzle: from a few examples, it works out the hidden rule that says which objects are wudsy. Impressive.
The road ahead
- 1 · The problem ~6 min One correct answer on one puzzle does not show whether a reasoning model has the general ability or just handled that exact puzzle.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Build variants of a solved task that keep its hidden structure but change surface details, and see whether the model solves them all.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Take rule induction tasks from cognitive science, generate structure-preserving variants of each, and check whether models solve every variant.
- 6 · The experiment ~6 min You’ll guess: When a reasoning model correctly solves a rule induction task, does it reliably solve structurally equivalent variants of that same task?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.