PatchHolmes: Agentic Patch Retrieval via Listwise Selection

PatchHolmes. Agentic Patch Retrieval via Listwise Selection. By Guanqun Yang and four others, from the Stevens Institute of Technology. Published on the thirtieth of September. Its code repository is worked out: about sixty source files, with a readme and a licence.

Patch retrieval means finding the commit that fixes a known vulnerability. It sits underneath vulnerability management workflows, and yet in the major advisory databases, between sixty and sixty-three percent of CVEs have no patch link at all. Prior work on the task has been pointwise: it scores each candidate commit on its own, independently of the others.

PatchHolmes is a two-phase system. A hybrid first-stage retriever narrows the field, and an agentic second-stage inspection loop makes the final call. The change from prior work is in how that agent looks at candidates. It reads the top one hundred listwise, seeing the whole candidate list at once instead of judging commits one at a time.

With the list in view, the agent reads selectively. It opens somewhere between three and ten commits, using four tools that each come with a budget, and then submits a single best commit. The entire system runs on a frozen open-weight model over a local Git repository. There is no fine-tuning and no external search service.

On GitHubAD, PatchHolmes beats the pointwise binary classifier Favia by twenty-five point three four percent in recall at one, and the retrieve-and-reason baseline IRCoT by thirty-one point four percent. It does so with one agent conversation per CVE, where Favia uses ten.

Two controls isolate where the gain comes from. Holding the candidate set fixed, the agent adds twenty-seven point three two percent in recall at one over simply taking the retriever's top candidate. Carried over unchanged to the PatchFinder top-ten candidate set, the same agent lifts recall at one from twenty-four point two eight percent, which is PatchFinder's own top pick, to thirty-nine point eight six percent.

Changing the backbone within the Qwen family moves recall at one by less than one percent. A second model family, gpt-oss, still stays far above the floor set by using no agent. The authors conclude that the improvement comes from the listwise agent loop rather than from any particular model.

A good detective, like the system's namesake, does not interrogate every suspect with equal care. PatchHolmes surveys the whole line-up first, chooses which few to question, and names one. The backbone experiments are what back this up: swap the backbone within a family and the result barely moves, so the credit goes to the way the search is organised.