MemT-1.0: Towards Memory-Native Evolution
Zhiyu Li, Zeyu Zhang, Linyun Liu, Yiwei Bu, Chenyang Xi, Xichong Zhang, Yi Yu, Yu Yu and 66 others
MemTensor
Published October 19, 2026 · on the feed October 20, 2026 · ▲ 135 · arXiv 2610.26825
The abstract
Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86times end-to-end inference speedup and a 66times reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.
The Brief
2:32A short podcast summary of the paper. Written and voiced by AI.
Transcript
MemT-1.0. Towards Memory-Native Evolution. By Zhiyu Li and seventy-three others, from MemTensor. Published on the nineteenth of October. Its code repository is worked out: about ninety-five source files, with a readme and a licence.
Memory-native evolution, as the authors frame it, requires making memory an intrinsic capability of foundation models, so that a model can continuously accumulate information and put it to use. The abstract measures itself against three existing ways of holding a history: parametric memory baselines, retrieval-augmented methods, and keeping the full context in a KV cache. Each of them keeps the history somewhere. The aim here is for the model's own computation to be where it is kept, and for that to scale to long histories without the cost of carrying everything along.
MemT-1.0 is a new architecture of memory foundation models, which internalize memory as part of model computation. It builds on Metis, and employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. To scale how much the model can natively remember, the authors further introduce Mixture-of-Storage.
Training starts with a deliberate preparation of the backbone, followed by a two-stage pipeline: continued pre-training, then post-training. It is supported by thirty-four billion tokens of training data, along with efficient infrastructure for training and for inference. The release is a model family with two members, MemT-1.0-Nano and MemT-1.0-Flash.
The evaluation is comprehensive, covering memory and general capabilities. On memory, the models outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving their general capabilities. Efficiency is measured at a history length of one hundred and twenty-eight thousand tokens. There, against a full context KV cache, MemT-1.0-Nano achieves approximately one point eight six times end-to-end inference speedup, and a sixty-six times reduction in persistent storage.
Memory also pays off on agentic tasks. By internalizing retrieved procedural experience, MemT-1.0 improves task success there.
Retrieval looks a history up, and a KV cache keeps all of it in full. MemT-1.0 takes a third route and folds the history into the model's own computation. On the reported results that route holds its own against retrieval on memory, and comes out far lighter than a full cache at long histories, which the authors present as a practical step toward memory-native evolution.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Every long conversation leaves a trail. The question is where a model should keep it.
You run an assistant that talks with the same users for months. Every past message is history it should remember and use.
The road ahead
- 1 · The problem ~6 min A long-running assistant must keep months of history, but every usual place to keep it costs more as the history grows.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min MemT-1.0 keeps memory inside the model's own computation as a fixed-size state, written token by token, so storage stays constant.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Prepare a strong backbone, teach it language modeling through native memory, then post-train it on memory tasks while keeping general skills.
- 6 · The experiment ~6 min You’ll guess: At a history length of 128K tokens, how much smaller was MemT-1.0-Nano's persistent storage than a full context KV cache?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.