MemoryAthena: Adaptive Routing over Latent and Generated Memories

MemoryAthena. Adaptive Routing over Latent and Generated Memories. By Mingyuan Li and six others, from the ELLIS Institute of Finland, the University of Turku, and three other institutions. Published on the twenty-second of September. Its code repository is worked out: about one hundred and sixty source files, with a readme and a licence.

Learned-memory methods keep information in an explicit table and read it through a separate reader. That separation lets addressing, storage and reading each be modified independently. The question the authors study is whether useful memory has to be retrieved from such a table, or whether it can also be generated. Retrieval can only return what was stored. Generation could, in principle, produce a memory suited to the moment, but it could also produce one that gets in the way.

MemoryAthena runs three pathways. The first is direct Engram retrieval, labelled E. The second generates memory from retrieved Engram cues, labelled GE. The third generates memory from the backbone's causal states without consulting the memory table at all, labelled GH. The difficulty is that generated memory is only conditionally useful. In one context it complements direct retrieval; in another it interferes with it. So MemoryAthena treats direct retrieval as the anchor, and learns when a generated representation should intervene.

Almost everything is frozen: the backbone, the memory, the generators, and the readers. The only part trained is a lightweight causal routing head. Its training signal is counterfactual, the advantage in future-token likelihood that each generated pathway has relative to direct retrieval. At inference, a candidate the router admits modifies the direct pathway's residual through bounded interpolation. A rejected candidate leaves the direct pathway exactly as it was. The whole memory-side system has about two hundred and one million parameters, not counting the frozen backbone. The consequence of freezing so much is safety by construction. The router can only choose whether and how far to move away from direct retrieval, and saying no always returns the original behavior.

On question answering, MemoryAthena raises the five-task average from thirty- seven point six five to thirty-nine point two eight, against the direct pathway of the same checkpoint. On general language tasks, the six-task average goes from seventy-six point seven three to seventy-nine point one three. Further analyses find complementary strengths among the three pathways, varying across tasks and across inputs. No single pathway wins everywhere, which is the reason a router is needed at all. If one of them always helped, it could simply be switched on.

The picture that emerges is generated memory as a selective correction to direct retrieval, not a replacement for it. Retrieval stays the default, and it can always be recovered exactly. The central challenge, as the authors frame it, is the routing itself: deciding when to intervene, which generated pathway to use, and how strongly.