TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization

TextReg. Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization. By Lucheng Fu and six others, from the Georgia Institute of Technology and the University of Illinois Urbana-Champaign. Published on the fifth of October. Its code repository is public but slight: seven source files.

Large language models are highly sensitive to their prompts, the text that specifies what the task is and which constraints apply. Many recent prompt optimization methods respond by rewriting the prompt again and again, guided by feedback that a language model generates. The prompts that come out of that loop tend to share a pattern. They grow longer, they collect narrow rules tied to particular samples, and they generalize poorly once inputs move beyond the training distribution.

The authors name this failure prompt distributional overfitting, and argue that its root is a lack of representation control when optimization happens in discrete text space. To make that precise they define representational inefficiency, a dual-factor measure. It splits a prompt's inefficiency into capacity cost and scope narrowness, and attributes the overfitting to those two growing together as optimization proceeds.

TextReg is the regularization framework that follows from this view. It realizes a soft-penalty objective through regularized textual gradients, and it combines three components: Dual-Evidence Gradient Purification, Semantic Edit Regularization, and Regularization-Guided Prompt Update. The abstract names these parts without detailing their internals, so the shape to hold on to is the overall one: the textual gradients that drive prompt rewriting are regularized before they are acted on.

Evaluation spans multiple reasoning benchmarks, and the target is out-of-distribution generalization. There TextReg improves substantially, with accuracy gains of up to eleven point eight percent over TextGrad, and up to sixteen point five percent over REVOLVE.

The contribution runs from diagnosis to remedy. Prompts optimized by feedback loops swell and specialize, a measure built from capacity cost and scope narrowness gives that drift a name, and regularization keeps it in check. The payoff is measured where overfitting would hurt most: on data from outside the training distribution.