Glance: The Generator Is Its Own Reranker

Glance. The Generator Is Its Own Reranker. By Adilet Metinov and one other, from Airun Intelligence Labs. Published on the twenty-ninth of October.

Retrieval-augmented generation commonly serves two separate models: one to rerank candidate passages, another to answer. That duplicates work twice over. Two models have to stay resident, and the passages get processed by each of them. The reranker reads the candidates to decide which matter, and then the generator reads the chosen ones again to answer. The same text passes through two sets of weights, on two footprints of memory, for one question.

Glance asks the generator to do the reranking itself. It equips the generator with a lightweight decision adapter and a pointer head, which score candidate passages from a shared prefill, so the passages are processed once and both jobs draw on that.

Short relevance probes branch off from the cached context, and each produces one score per passage, without a separate prefill for every query and passage pair. A single resident nine billion parameter backbone then covers generation, general decisions and reranking, at roughly zero point one gibibytes of additional memory for each decision skill. The expensive part, reading the passages, happens once. What is added per passage is only a short probe, and what is added per skill is only a small adapter.

After thirty-nine minutes of fine-tuning on MS MARCO, Glance reaches an nDCG@10 of zero point seven four one on SciFact, against zero point seven two one for bge-reranker-v2-m3.

On NFCorpus it scores zero point three four five, against zero point three two six. On three hundred HotpotQA questions, the context it selects improves answer F1 by zero point zero four five over cross-encoder-selected context, with the same base generator, and a paired bootstrap confidence interval that stays above zero.

In a scaling study, reranking quality keeps rising up to two hundred candidates, about forty-three thousand tokens, all scored from one shared prefill. At that point it beats the cross-encoder by zero point zero five eight nDCG@10. A probe-only activated adapter also lets generation continue from the original passage cache.

The authors present these as initial results, and are explicit about the coverage: two retrieval datasets, one QA dataset, and one backbone family. Within that, the demonstration is that reranking need not be a second model, only a short question put to the one already holding the passages.