Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks
Reza Esfandiarpoor, Radek Osmulski, Yauhen Babakhin, Gabriel de Souza P. Moreira, Oliver Holworthy, Jie He, Ronay Ak, Jiarui Cai and 3 others
NVIDIA · University of Edinburgh
Published October 5, 2026 · on the feed October 6, 2026 · ▲ 18 · arXiv 2610.05750
The abstract
Modern information systems, including many agentic workflows, use dense retrieval to explore large amounts of unstructured data. However, dense retrieval relies on surface-level semantic similarity, which is insufficient for increasingly complex search applications. Here, we investigate agentic retrieval that combines the reasoning capabilities of Large Language Models (LLMs) with the efficient corpus exploration of retrievers in a ReAct agentic loop to solve complex retrieval tasks. In our experiments, we show that agentic retrieval is more effective than standard retrieval, improving nDCG@10 by 8.7 points using the same embedding model. Moreover, while specialized retrieval methods struggle on out-of-domain tasks, agentic retrieval is highly generalizable: the same pipeline achieves competitive results on both the ViDoRe v3 and BRIGHT leaderboards. However, this improvement comes at a cost. On average, agentic retrieval takes 107.4 seconds, compared to 0.67 seconds for standard retrieval, and consumes 764.1K input and 5.8K output tokens per query. In short, our study demonstrates the effectiveness of agentic retrieval in modern data systems and motivates future work on more cost-efficient retrieval agents for large-scale deployment.
The Brief
1:59A short podcast summary of the paper. Written and voiced by AI.
Transcript
Beyond Semantic Similarity. Performance and Costs of Agentic Retrieval for Complex Tasks. By Reza Esfandiarpoor and ten others, from NVIDIA and the University of Edinburgh. Published on the fifth of October. It links to a code repository that is no longer there.
Dense retrieval underpins many modern information systems, agentic workflows included. But it relies on surface-level semantic similarity, and that is not enough for increasingly complex search. This study puts the reasoning of a large language model in charge of a retriever, inside a ReAct agentic loop, and asks two questions. Is agentic retrieval more effective than standard retrieval? And what does it cost? The answer to the first is yes, by a clear margin. The answer to the second is a great deal.
Using the same embedding model, agentic retrieval improves nDCG at ten by eight point seven points over standard retrieval. It also generalizes where specialized retrieval methods struggle. Those methods falter on out-of-domain tasks, while the same agentic pipeline achieves competitive results on two different leaderboards, ViDoRe v3 and BRIGHT.
The cost side is stark. On average, an agentic query takes one hundred and seven point four seconds, against zero point six seven seconds for standard retrieval. Each query also consumes about seven hundred sixty-four thousand input tokens, and five point eight thousand output tokens.
The claims are bounded by what was run. The effectiveness gain is measured against standard retrieval with the embedding model held fixed, and generality is shown through the two leaderboards named. Costs are reported as averages per query. The abstract does not offer a cheaper variant: it frames cost-efficient retrieval agents as future work that large-scale deployment will need.
Two numbers sit next to each other here and neither cancels the other. Ranking quality goes up by almost nine points, and a query that took well under a second now takes nearly two minutes. The authors present agentic retrieval as effective for modern data systems, and the same evidence is their case for building retrieval agents that cost less to run.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free