Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks

Memento 3. Model-Based Recursive Self-Improvement through Reflective Rulebooks. By Haoyu Zhao and seven others, from University College London, Huawei Noah's Ark Lab, and one other institution. Published on the eighth of October.

An agent dropped into an unfamiliar environment has to work out how the world behaves, and revise that picture as new evidence comes in. The hard part is ambiguity. A handful of observations can be explained equally well by several different world models, which all agree on the past and yet predict different outcomes in states the agent has not visited. An agent that commits too early to one of them will act confidently and be wrong the moment it steps somewhere new. An agent that never commits cannot plan at all. Learning in this setting means holding hypotheses loosely, and revising them as soon as the evidence turns against them.

Memento 3 builds on the earlier Memento series. Its aim is to let a frozen language model agent keep learning an explicit world model, through external memory. The agent maintains a rulebook written in natural language as persistent semantic memory. The rulebook records revisable hypotheses about how the environment works, and it deliberately leaves unknown aspects underspecified instead of guessing. The agent then compiles that rulebook into executable code, which it uses for prediction and for planning.

Learning happens in a continual loop: observe, reflect, revise the rules, compile, verify. Prediction errors drive the refinement of both the rulebook and its code. Acceptance is strict. Updated code is kept only when two checks pass: the language model judges the code faithful to the rulebook, and a cell-exact replay reproduces the transitions that were actually observed. The first check keeps the code tied to the rules the agent believes. The second keeps both tied to what really happened. A change that fails either one is rejected, so the world model only moves when it moves toward the evidence.

The authors frame this as a model-based route to recursive self-improvement. The agent explores on its own, revises its world model, and uses verified updates to steer what it does and learns next, while the underlying model never changes. A population extension goes further, keeping several world models in parallel. They share interaction evidence, and their predictions are used to guide exploration, which speaks directly to the problem of many models fitting the same past.

On ARC-AGI-3, the single-model agent clears every level of all twenty-five public games. It achieves a mean Relative Human Action Efficiency of one hundred, and uses forty-four percent of the human action count.

In a case study on Atari Pong, a learned feedback controller wins twenty-one to zero in each of three evaluated episodes, each with a different opening, and it does so with no further calls to the language model.

Self-improvement here does not mean changing weights. It means a model that stays fixed while its understanding of the world, written down as rules and compiled into code, gets better and is checked at every step. The Pong result shows where that leads: once the world model is right, the agent can act on it without asking the language model anything at all.