Recurrent Harness for Agentic Continual Learning
Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang
Massachusetts Institute of Technology · Meta MRS Research · Yale University
Published October 8, 2026 · on the feed October 12, 2026 · ▲ 23 · arXiv 2610.12668
The abstract
Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.
The Brief
2:06A short podcast summary of the paper. Written and voiced by AI.
Transcript
Recurrent Harness for Agentic Continual Learning. By Yuxin Xiao and six others, from the Massachusetts Institute of Technology, Meta MRS Research, and one other institution. Published on the eighth of October. No code repository is linked to it.
Language model agents increasingly work through long streams of interaction, and the distribution of what they meet shifts along the way. Keeping up means adapting online, which is a continual learning problem. The harness around an agent, the state it reads from and writes to, is where that adaptation tends to live. Existing harnesses fall into two camps. Some only ever move information forward, writing new state and never revisiting it. Others optimize the harness with the help of extra replay or validation data.
ReHarness is a recurrent harness framework that adds a third direction. Alongside forward reading and writing, it performs backward revision, and it does so under a strictly online streaming protocol, with no extra data held aside. The guiding question is a counterfactual one: what could the harness state have captured differently that would have better informed the instance the agent just faced?
The mechanics follow from that question. After each instance in the stream, the trajectory it produced is used as feedback. That feedback revises the reflections that were retrieved for the instance. Because those reflections are part of the harness state, which is written in text, the feedback is in effect propagated backward through the textual state, much as an error signal is passed back through a recurrent network.
The evaluation covers three continual learning benchmarks and three language model backbones. ReHarness consistently outperforms strong raw-context baselines, methods built on memory and reflection, and prompt optimization approaches. Further analysis finds that its revisions are source-grounded and meaningful, that they correct experience which had become overgeneralized or outdated as the distribution shifted, and that the approach improves both stability and plasticity.
Stability and plasticity usually pull against each other: holding on to the old versus taking in the new. Revising past reflections in light of what just happened lets a harness do some of both, keeping lessons while rewriting the ones that no longer hold.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free