Agent Plasticity: Measuring Self-Improvement Through Experience

Agent Plasticity. Measuring Self-Improvement Through Experience. By Harman Singh and ten others, from UC Berkeley, Meta Superintelligence Labs, and two other institutions. Published on the sixth of October.

AI agents increasingly work in environments where they can diagnose their own failures and improve through experience. Evaluation has not caught up with that. Existing evaluations largely measure what an agent can do at a fixed point in time. They do not measure how effectively it learns. A snapshot of capability says nothing about the slope, and for an agent that is supposed to get better, the slope is the thing you would want to know.

The authors break self-improvement into three questions. First, does future performance actually improve, and does it generalise beyond the interactions that enabled the learning? Second, how efficiently are new capabilities acquired? Third, where does the self-improvement process break down? The central quantity they introduce is agent plasticity: the efficiency with which an agent converts experience into gains in future performance on held-out interactions.

To answer those questions, self-improvement is studied in a controlled setting. Agents amortise past experience into reusable artifacts, and those artifacts are inherited by future instances of the agent. Learning therefore has a concrete form that can be inspected. At each checkpoint, performance is measured on both the training interactions and held-out environment interactions, and the cost of learning is accounted for, so that gains are weighed against what they took to acquire.

Each part of that setup lines up with one of the three questions. Measuring on held-out interactions, and not only on the ones the agent learned from, is what tests whether improvement generalises. Accounting for cost at every checkpoint is what turns improvement into an efficiency, which is the sense in which plasticity is defined. And because what the agent learned is stored in artifacts that later instances inherit, a breakdown can be traced to a place in the loop, rather than inferred from a final score.

Across multiple environments, frontier models follow sharply different improvement trajectories, despite comparable opportunities to learn. Some achieve substantial and persistent gains. Others stay near their initial performance, or fall below it. Gains made within the training regime often transfer only partially to out-of-distribution conditions.

Two further results separate ideas that are easy to conflate. Endpoint capability and acquisition efficiency diverge: the agent that performs best in the end need not be the one that improved most efficiently. And tracing failures through the improvement loop points to different candidate bottlenecks for different agents. Agents with low plasticity often fail to reuse relevant artifacts at all. More plastic agents may reuse the relevant artifacts and still fail, which points instead to limits in artifact quality, in generalisation, or in how the artifacts are applied.

That last distinction gives the measurement its diagnostic edge. Two agents can fail to improve for opposite reasons, one never picking up what it learned and the other picking it up and misusing it, and only an evaluation that follows the loop can tell them apart. The authors conclude that judging self-improving agents means measuring not only what they can do, but how effectively they become better through experience.