ASCENT: Online Test-Time Training of Long-Horizon Agents via Self-Distillation of Verified Experience

ASCENT. Online Test-Time Training of Long-Horizon Agents via Self-Distillation of Verified Experience. By Haodong Lu and one other, from the University of New South Wales. Published on the fourth of October. No code repository is linked to it.

An agent built on a large language model solves a long-horizon task over many turns of reasoning and action, and gets a single verification signal only at the end. Once deployed, it faces a stream of related tasks, and its own trajectories become a natural resource for getting better.

The common way to use them is in-context adaptation: the agent stores reflections, memories, or skills as text. That makes reuse depend on two things going right. The right experience has to be retrieved, and a frozen policy has to execute it. Neither is guaranteed.

The authors study an alternative they call Online Agentic Test-Time Training, or OaTTT, where the model's weights are trained on its own execution trajectories during deployment. The setting is strict. The agent executes each task once, in one pass over the stream. The executed trajectory, with its verification result, is the only learning signal, and the weight updates it produces persist across tasks. Here is the difficulty. In this setting, directly imitating or reinforcing the tokens generated in that single attempt destabilizes the policy.

ASCENT, short for Agentic Self-distillation for Cross-task EvolutioN at Test-time, replaces imitation with self-distillation of verified experience. The teacher is not a stronger model and not a reference solution. It is the agent's own frozen initial copy, looking back at the trajectory with hindsight. Because that copy never changes, it gives the updates a fixed point of reference.

That stable copy receives the verified trajectory as privileged information, and with this hindsight it predicts next-token distributions along the trajectory. Those distributions are distilled into persistent LoRA fast weights, which update the agent for the tasks that come later. No external reference solution and no stronger teacher are needed: everything comes from the agent's own attempt and the verdict on it.

A refinement makes the experience more efficient to learn from. ASCENT removes turns containing invalid actions before distilling, so what it learns from is an enhanced version of the privileged experience, and the agent is pushed toward more efficient execution. The authors also characterize the population target of this procedure, and the limits of sparse outcome selection.

Experiments cover ALFWorld, WebShop, and AppWorld, at varied model scales. As experience accumulates, ASCENT improves both task success and interaction efficiency. It outperforms online adaptation methods, and it transfers to held-out scenes. The abstract reports these outcomes without figures.

Think of it as moving experience from the prompt into the weights. Instead of storing what happened as text and hoping it is retrieved and followed, the agent consolidates verified experience directly, with no separate training phase and no memory retrieval. Stability comes from the choice of teacher: the original model, reading the trajectory with the benefit of knowing how it ended.