Self-Preservation in Language Models

Self-Preservation in Language Models. By Dongkyu Cho and two others, from New York University, Google Research, and one other institution. Published on the eighth of October. Its code repository is a placeholder: a readme, and nothing to run yet.

Continual learning tries to adapt a pretrained model to something new while keeping what it could already do. The trouble is coverage. The tasks handed over for preservation describe only part of what a model knows, and forgetting does not politely stay inside that boundary. Earlier work has shown that a language model can fill some of this gap by generating its own replay data, with no access to the data it was originally trained on. That opens a sharper question, which the authors call self-preservation. Can a model help decide what should be preserved, so that its own capabilities are protected more broadly than the supplied tasks alone would allow? Their answer is a qualified yes: models can support broader self-preservation by generating replay and, in some settings, by choosing which tasks to replay.

To study this, task descriptions and their examples are organised into Task Banks. Entries in a bank can be selected and combined, and they guide replay generation by a frozen model. The experiments are continual instruction tuning with Qwen models of several sizes, and they produce three findings.

The first concerns what gets lost. Which prior capabilities degrade depends on the new task and on the learning rate. Tracking whether previously correct answers stay correct exposes losses that an average accuracy figure can hide.

The second concerns breadth. Generated replay reduces forgetting even on tasks that are not in the bank. Combining partial banks makes that protection stronger: across the tested sizes, mean retention of previously correct answers beats either bank used alone.

The third concerns choice. The model can help pick its own replay targets. When the selection is informed by the new task, mean held-out retention improves over random selection at the four billion parameter size, and the gains on the new task stay similar.

The limits are stated plainly. The advantage of model-informed selection appears at four billion parameters, and smaller models show no corresponding advantage. So the ability to choose well is not something every size inherits. There is also a trade-off under a fixed example budget. If replay examples take the place of new-task examples, preservation improves, but the new task is learned less. Protection here is not free; it is paid for out of the same budget. And the evidence comes from one model family, Qwen, under continual instruction tuning, so the findings describe that setting rather than continual learning in general.

Picture the model as a curator of its own memory. Handed a partial list of what to protect, it can write replay that guards more than the list names, and mixing partial lists guards more still. Past a certain size it can even suggest what belongs on the list. The idea carrying all three findings is that the evidence for preservation need not come only from outside: some of it can be generated, and some of it chosen, by the model being preserved.