A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt

A Conditional No-Hair Theorem for Transformers. How Language Models Forget Their Prompt. By Shuyang Xiang, working independently. Published on the sixteenth of October. Its code repository is public and real: fifteen source files.

The question is how much an autoregressive transformer still carries of its prompt after it has generated some number of tokens, call that number t. The experiment that frames it is clean. Take two different prompts, and follow each with exactly the same continuation text. Any difference between the two runs' hidden states can then only come from the prompts.

Measured this way, the distance between the hidden states decays roughly as a power law in t, with an exponent called alpha. An alpha of one is a natural reference point: it is the rate at which a fixed prompt's share of uniform attention shrinks as the context grows. Smaller values of alpha mean longer memory. Trained models forget quickly, with alpha around zero point five five to zero point seven. Untrained models forget slowly, with alpha around zero point two to zero point five.

The paper proves a bound on that offset. For a prompt of a given length, and a network of a given depth, the offset is at most of order the prompt length, times the logarithm of t raised to the power of depth minus one, all divided by t. In words: it falls off like one over t, slowed down only by logarithmic factors whose number grows with depth.

The author calls this the conditional no-hair theorem. Given the text generated so far, the prompt's influence on the next token vanishes as t grows. So any property of the prompt that does survive, its language for example, has to be carried by the generated text itself, not by some hidden residue of the prompt.

The mechanism is geometric, and it has two parts. The first is dilution. The prompt's direct influence is spread thinner as the context gets longer, simply because it becomes a smaller share of what is being attended to. The second is relay. Continuation tokens can pass on what earlier tokens read from the prompt, so the prompt can reach the present indirectly. The argument shows that relay can only add logarithmic factors on top of dilution. It cannot undo it.

Two further pieces support the picture. An exactly solvable linear relay model shows the bound cannot be improved, and it reproduces the rise of the local exponent towards one that is seen in untrained networks. Then comes an intervention. Cancelling the dilution with a logit bias stops the decay early in training, and reverses it later in training. So dilution is necessary for the forgetting that is measured. The untrained networks and the solvable model show that it is also sufficient.

The bound is stated for a specific class of network: finite-depth, pre-layer norm, softmax transformers, with attention logits that stay bounded. On top of that sits one additional condition, and it is the reason the theorem is called conditional. The activations going into each normalisation must stay away from zero.

Put plainly, under those conditions a transformer keeps less and less of its prompt beyond what the text it has written already says. Whatever persists lives in the visible continuation. And the paper pins the cause down from both directions: remove dilution and the forgetting stops, keep only dilution and the forgetting appears.