ABF-Bench: A Verifiable Benchmark for Agentic Blockchain Forensics

ABF-Bench. A Verifiable Benchmark for Agentic Blockchain Forensics. By Xiaohang Yu and two others, from Imperial College London and the University of Oxford. Published on the sixteenth of October.

Blockchain forensics is adversarial and open-ended. Money launderers split, swap and bridge funds. They cut their trail at mixers or custodial services. And they change tactics from one incident to the next, so investigators can never settle on one fixed procedure: they have to keep adapting how they trace. There is a counterweight, though. Public ledgers are immutable, and every recorded movement stays in public view, which makes much of an investigation independently verifiable. A task that is hard and open-ended, but whose answers can still be checked against a public record, is an unusual combination, and it is the combination this benchmark sets out to use for agents.

ABF-Bench is an interactive benchmark that turns evidence-backed fund tracing into a verifiable agent task. The rule at its centre is strict. Every movement of funds that an agent claims has to cite ledger records, and not just any records: ones the agent itself actually retrieved during the task. An agent cannot assert a transfer from memory or from a guess. What gets scored is the trail it reconstructs, compared against a reference graph, with edge recall as the headline number and fidelity reported alongside it.

Tasks come from a case-to-task pipeline that builds well-posed problems across ledgers. The pipeline uses conservative rules that halt tracing wherever the records leave a continuation unresolved, and each task's reference graph is simply the answer those rules determine. That is what makes an open-ended investigation into something with a definite answer. The tasks are built from real laundering activity rather than invented cases. They combine multi-hop transfers, DeFi exchanges, cross-chain round trips, deposits to custodial services and to mixers, and address poisoning.

Alongside the benchmark, the authors propose their own agent, the ABF agent harness, which is built around a working memory. They evaluate it next to two other kinds of tracer: general-purpose agents, and fixed-order tracers.

The harness reaches seventy-five percent edge recall. Codex, running the same model, reaches forty-seven percent. Claude Code reaches ten percent, and the reason given is overconfidence: it cites records it never retrieved, which is exactly what the verification rule forbids.

Against fixed-order tracers the picture is mixed. Early in the budget, the harness acquires evidence faster than the best of them. It recognizes undisclosed custodial services that the tracers walk straight past, eighty-nine percent against zero, with ninety-eight percent fidelity. But agents tend to stop early, and they lose recall on larger tasks. That tendency holds their final recall below the best fixed-order tracer, even though they led early on.

Beyond the gaps it exposes, ABF-Bench presents blockchain forensics as a domain where LLM agents can be checked rather than trusted. Because every verdict comes from the ledger, the same verdicts that score an agent could, the authors suggest, also serve as training signals. The record that the agent searches is also the record that grades it.