MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization

MemFold. Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization. By Jingxuan Wu and eight others, from the University of California, Santa Barbara, the University of North Carolina at Chapel Hill, and two other institutions. Published on the twenty-ninth of September. Its code repository is worked out: about seventy-five source files, with a readme and a licence.

Picture an assistant serving the same person for a long time. To answer well it has to draw on what that person has revealed: which preferences still hold, which ones were revised, and which constraints apply right now. The authors draw a line that is easy to blur. Retaining that information is one thing. Acting on it is another. Yet the two are usually optimized as if they were the same.

The usual options each carry a cost. Keep the history as text, and the reader's input grows with everything retained. Compress it into a fixed number of latent vectors, and the interface is bounded, but such compressors are typically trained either to reconstruct text or to imitate reference answers. Both of those objectives are scored on sequences the reader itself never produced. The memory is graded on something other than what the reader does with it.

MemFold optimizes a fixed-budget soft memory by the behavior it supports. A textual memory, conditioned on the query, is compressed into K continuous vectors, and those vectors are the reader's whole memory interface. However much history the user has built up, the reader sees the same K vectors, so its input stays bounded. Then the reader is trained on its own rollouts. The training signal comes from what it actually generates, rather than from reconstructed text or from someone else's reference answers. Retaining and acting are no longer optimized as separate goals: the memory is judged by the behavior it makes possible.

Two complementary signals drive that training. The first is group-relative rewards for task outcomes: did the answer get it right. The second is confidence-gated on-policy distillation. Here a frozen teacher, which has the full textual memory, re-scores the tokens the student sampled, under that textual memory.

The teacher is never sampled from. That keeps supervision on the student's current distribution, and it means the teacher adds no autoregressive decoding of its own. At inference the teacher is removed entirely, and only the reader and its compact memory remain.

Testing spans three Qwen backbones. On PersonaMem-32K and PersonaMem-128K, MemFold attains the highest accuracy among the systems measured, and its margins widen at the longer history length. The gap grows, that is, as there is more history to compress. It also transfers to PrefEval and to LongMemEval without any training in those target domains.

The ablations split the credit. Most of the task gain is attributed to the reward term, and a smaller additional gain to the teacher signal. Memory interventions add one more check: the reader depends on the instance-specific content of its soft memory. What is in the vectors for this particular user is what the answers rest on.

Compressing memory is the familiar half of this. The shift MemFold makes is in what the compression answers to. Instead of rewarding a memory for preserving text, it rewards the memory for the answers the reader produces from it, on the reader's own samples, and keeps a text-reading teacher only as long as training lasts.