Can Computation from Earlier Problems Help LLMs Solve New Ones?

Can Computation from Earlier Problems Help LLMs Solve New Ones?. By Jipei He and six others, from Renmin University of China and the University of Modena and Reggio Emilia. Published on the thirtieth of September. No code repository is linked to it.

Large language models often solve independent problems inside a single conversation, one after another. That raises a plain question: can the computation spent on earlier problems help with a new one? Preliminary experiments give an unsettled answer. Retained history can raise later-turn accuracy, and it can also lower it, even when every problem comes from the same domain. Keeping the history around is not a reliable gift.

To see where those effects come from, the authors use controlled replay, which isolates the internal state changes specific to each pairing of a problem with a history. Across different histories, those changes preserve similar relationships among the current problems. With that picture in hand, they introduce STAIR, short for Stale-Token Attention for Inter-query Reuse, a way to improve reasoning when the history is kept.

STAIR captures the keys and values produced while generating earlier responses, and holds them in a fixed bank. When the current queries read that bank during prompt processing, STAIR learns to redirect them. Everything else stays put. The base model remains frozen, and only twelve thousand two hundred and eighty-eight parameters are trained.

The evaluation spans three Qwen models and four benchmarks. Measured against the unmodified model carrying the same history, STAIR improves average later-turn accuracy by up to eleven point six seven percentage points. The comparison is a fair one to keep in mind: both sides see the earlier problems, and only one of them has learned how to read what those problems left behind.

Conversation history is double-edged here. It can help a later problem or hurt it. Rather than discarding it, or retraining the model to cope, STAIR treats earlier computation as a stored resource and trains a very small component to decide how current queries should read it.