Remembering Has a Price: Learning Memory Worth under Resource Constraints

Remembering Has a Price. Learning Memory Worth under Resource Constraints. By Guannan Lai and three others, from Nanjing University. Published on the nineteenth of October. Its code repository is worked out: about one hundred and thirty source files, with a readme.

Lifelong agents keep accumulating experience after deployment and reuse it to get better. In practice, though, they usually run under tight budgets on three fronts at once: how much they can generate, how much they can interact, and how much they can store.

Under those joint constraints, the authors find that existing lifelong-memory methods can stop working. Several sophisticated approaches end up offering little advantage over simple retrieval, and sometimes little over an agent with no memory at all.

Behind that result is a more basic problem. When resources are scarce, every memory decision carries more weight. Only a small amount of past experience can be kept or reused, and there are few chances to reason further or to recover from a mistake. In that setting, retrieving a memory that is relevant but not actually useful does real harm. It uses up scarce context and gets in the way of what the agent does next.

The real challenge, then, is not just finding relevant experience. It is deciding which experiences genuinely deserve to be kept and reused. The paper calls this memory worth, and treats it as something to be learned. TokenWorth is a lightweight online memory framework that learns the worth of a memory with respect to the current query.

The signal it learns from is sparse feedback from execution, gathered online as the agent works. The worth it learns is then applied consistently in two places. It shapes memory retrieval, deciding what is brought into context for a given query. And it shapes bounded memory management, deciding what stays in a store that cannot grow without limit. One learned quantity governs both what is recalled and what is kept.

Evaluation covers three LifelongAgentBench environments, under both local deployments and deployments that call models through an API. Across these, TokenWorth achieves the highest task success rate of the methods evaluated, and it does so while using the fewest output tokens among them. Success went up while spending went down, in a setting where spending is exactly what is scarce.

The title puts it plainly: remembering has a price. When context, storage and chances to recover are all limited, a relevant memory can still be a bad buy. The results suggest that lifelong adaptation under those limits depends not on remembering more, but on learning which experience earns its place.