Towards In-Parameter Memory Augmentation for Large Language Models

Towards In-Parameter Memory Augmentation for Large Language Models. By Haoyu Huang and eight others, from the Hong Kong University of Science and Technology and Hong Kong Baptist University. Published on the sixth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Large language models, and the agents built on them, increasingly need knowledge acquired after pretraining. Domain facts, user preferences, documents, the experience of past interactions. The usual route is in-context learning, or an agent harness built around it. That route is flexible, but it consumes context capacity, and it pays a repeated cost of discretized encoding that grows with the length of the context.

In-parameter memory offers a complementary substrate. Reusable memory information is represented in model parameters, in adapters, or in other parameter-like objects, which are composed into the forward pass at inference time. This survey focuses on methods that augment a model with such parametric memory at deployment: a memory-bearing parameter object is plugged into the forward pass during inference, whether it was acquired before deployment or during it.

The landscape is organised along two orthogonal axes. The first is Parameter Placement: where in the model the memory lives. The options are the Embedding, Attention, or FFN layers, or Hybrid, when two or more of those are used together. The second axis is Parameter Acquisition Time. It separates methods whose memory object is acquired during deployment, called online, from those whose memory object is acquired before it, called offline. Any method can be located by its answer to both questions at once.

With that map in place, the survey clarifies boundaries and conducts comparisons across methods. The boundary-drawing matters because the scope is deliberately specific. The subject is not every way knowledge can end up in weights, but memory that is packaged as a parameter object and plugged into the forward pass when the model is actually serving.

The survey closes on open directions, and these name the pressures running through the whole area. Interference. Safety. Co-design with in-context learning, which fits the survey's framing of the two substrates as complements rather than rivals. And recursive self-improvement, which sits naturally beside the online half of the second axis, where memory is acquired while the model is already deployed.

The framing positions parametric memory not as a replacement for context but as its companion. Context stays flexible. Parameters hold what is reused, and two simple questions, where the memory sits and when it was acquired, are enough to sort the field.