Recognition Is Not Grounding: Benchmarking Multilingual Financial Evidence Extraction
Lingfei Qian, Yan Wang, Xueqing Peng, Polydoros Giannouris, Nadhem Zmandar, Yi Han, Yanguang Liu, Víctor Gutiérrez-Basulto
The Fin AI · University of Manchester · Georgia Institute of Technology · New Jersey Institute of Technology · Cardiff University
Published October 14, 2026 · on the feed October 19, 2026 · ▲ 6 · arXiv 2610.19432
The abstract
Comparing financial disclosures across languages requires identifying shared accounting concepts and recovering all numerical facts that express them. Existing benchmarks evaluate aspects of these capabilities, but provide limited insight into whether both succeed on the same multilingual evidence. In particular, it remains unclear whether language models can recognize these concepts across languages and exhaustively recover their corresponding numerical evidence from the same contexts. We introduce MULTIFINGROUND, a benchmark designed to investigate this recognition-grounding gap in multilingual financial reports. Constructed from FY2025 disclosures of 90 issuers in English, French, and Greek, MULTIFINGROUND aligns independently authored reports through a shared accounting taxonomy and provides traceable fact-level annotations. The benchmark evaluates financial fact identification, concept recognition, exhaustive fact selection, and concept discrimination without candidate labels. Across 15 evaluated models, eight API and open-weight reference models achieve 97.3-99.4\% concept recognition accuracy, yet miss at least one target fact in 26.6-41.9\% of correctly recognized groups. Further analyses reveal substantial variation in concept discrimination and limited gains from the evaluated finance-specialized models. Our findings demonstrate that strong concept recognition can coexist with incomplete evidence recovery, highlighting the importance of evaluating semantic recognition and evidence completeness together.
The Brief
2:10A short podcast summary of the paper. Written and voiced by AI.
Transcript
Recognition Is Not Grounding. Benchmarking Multilingual Financial Evidence Extraction. By Lingfei Qian and seven others, from The Fin AI, the University of Manchester, and three other institutions. Published on the fourteenth of October. No code repository is linked to it.
Comparing financial disclosures written in different languages takes two skills. You have to spot the shared accounting concepts. And you have to recover every numerical fact that expresses them, not just some. Existing benchmarks look at pieces of this, but say little about whether both succeed on the same multilingual evidence. Can a model recognise a concept across languages, and also exhaustively pull out its numbers from the same context? Getting the first right does not guarantee the second, and until now there was no clean way to check both at once.
MULTIFINGROUND is designed to probe that recognition-grounding gap directly. It evaluates four things: identifying financial facts, recognising concepts, selecting facts exhaustively, and discriminating between concepts without being handed candidate labels. That last condition matters, because without a menu of labels the model has to tell concepts apart on its own.
The source material is FY2025 disclosures from ninety issuers, in English, French and Greek. These reports were written independently, not translated from one another. The benchmark aligns them through a shared accounting taxonomy, and annotates them at the level of individual facts, each of which can be traced back to its source.
Fifteen models were evaluated. Eight API and open-weight reference models reach between ninety-seven point three and ninety-nine point four percent accuracy on concept recognition. That part looks nearly solved. Yet among the groups they recognise correctly, they still miss at least one target fact somewhere between twenty-six point six and forty-one point nine percent of the time.
Further analysis shows that concept discrimination varies substantially from model to model, and that the finance specialised models in the evaluation bring only limited gains.
The title says it plainly: recognition is not grounding. A model can name the right concept almost every time and still leave numbers behind, which is why the authors argue that semantic recognition and completeness of evidence should be evaluated together rather than one at a time.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free