Replacing Training with Memory: Listwise Selection for Text-to-SQL

Replacing Training with Memory. Listwise Selection for Text-to-SQL. By Yeonseok Jeong and three others, from Seoul National University and KAIST. Published on the first of September. Its code repository is public and real: nineteen source files.

Modern Text-to-SQL systems often work as a pipeline of three steps: generate several candidate queries, execute them, then select the best one. Listwise selection, where a model compares several candidates jointly instead of scoring each alone, has been widely adopted for that last step. The catch is training. Fine-tuning a listwise selector is costly, and the fine-tuning is doing two jobs at once. It teaches the model which criteria to order candidates by, and it teaches the model to resist positional bias, the pull toward a candidate simply because of where it sits in the list.

MaP-SQL keeps listwise selection and drops the fine-tuning. Each of those two training objectives is replaced with a strategy that runs at inference time. Selection criteria, instead of being learned into model parameters, are kept as reusable structured memories. Ordering bias, instead of being trained away, is averaged out.

The memories are distilled from training data. Given a question, MaP-SQL retrieves the ones that encode how natural language maps to schema elements, to SQL operations, and to expected outputs. Those memories then serve as explicit decision criteria when the model evaluates candidates side by side. For ordering bias, the method aggregates rankings across multiple permutations of the input, so that no single placement decides the outcome. Running many permutations could get expensive, and the abstract says that cost is kept down using execution results and pointwise scoring.

Across Text-to-SQL benchmarks, the approach produces more stable selection without fine-tuning, and makes fewer unnecessary comparisons than existing methods. The headline result is on BIRD-dev. Using the same candidate sets, it outperforms the previous state-of-the-art selector-based method by two point zero two execution accuracy points on average. It does so while using two point nine two times fewer tokens. The authors also stress that it stays efficient and compatible with existing large language models.

The title states the trade plainly. What a fine-tuned selector would have absorbed into its weights is here written down as memory that can be retrieved, and what training would have suppressed is handled by asking the same question in several orders. Selection accuracy goes up, and the selector itself never has to be trained.