MemSteer: Turning User Histories into Supervision for Agent Memory Optimization

MemSteer. Turning User Histories into Supervision for Agent Memory Optimization. By Jie Huang and four others, from Peking University and Microsoft Research Asia. Published on the twenty-ninth of October.

An agent's memory system works well when it adapts to the user's history and to what the task requires. Human-annotated supervision can guide that adaptation, by providing feedback for evaluating and improving memory designs. The difficulty is supply. Suitable labels are rarely available for a new user's history, and constructing question-answer pairs with supporting evidence by hand is costly.

MemSteer manufactures the supervision instead of waiting for it. It turns unlabeled user histories into question-answer supervision, complete with supporting evidence, that can then be used to optimize a memory design. The histories already exist for every user. What was missing was a way to read questions, answers, and the evidence linking them out of those histories, and to trust the result enough to steer a design with it.

Everything begins with an evidence-grounded answer plan, which specifies how facts spread across related records jointly support a reference answer. That plan guides question generation. Two checks are then applied to the resulting annotations: source-grounded auditing, and blind answer recovery.

To study whether this supervision is useful, the authors develop MemSteer-Opt. It combines question-based evaluation of a design with failure diagnosis guided by the evidence, and uses both to revise three things: the memory store, the retrieval, and the way context is constructed. No model parameters are updated along the way.

On ATM-Bench and RHELM, the generated supervision is competitive with optimization guided by the official question sets. Relative to Base-BM25, MemSteer-Opt improves the ATM-Bench Question Score by thirteen point four points. On RHELM it improves accuracy by four point one points without external data, and by six point six points with it.

The supervision also benefits two existing optimizers on ATM-Bench. And in experiments with new users, design optimization continues using supervision generated from those users' own target histories.

Labels were the bottleneck, and MemSteer's answer is to derive them from the very histories the memory system has to serve. A plan for each answer comes first, then the question, then two checks on the result. That ordering is what lets the questions be generated for any new user, without waiting for someone to write them.