Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

Think Before You Link. Rarity, Reasoning, and Retrieval in Multilingual Entity Linking. By Parinthapat Pengpun and two others, from Carnegie Mellon University. Published on the ninth of September. Its code repository is public and real: seventeen source files.

Multimodal entity linking grounds mentions of entities, in text and in images, to entries in a knowledge base. These systems are known to degrade on rare entities. The paper makes two claims about that weakness. The first is about measurement: how rare an entity counts as depends on how rarity is defined, and different definitions expose different failure modes. The second is about repair: reasoning and retrieval are complementary, and it takes both to help with the rare cases.

Earlier work has measured rarity mainly by popularity, with metrics such as pageviews. The authors widen that view with structural metrics from the knowledge graph, which capture how well an entity is documented and how well it is connected. These metrics pick out many rare entities that popularity misses. On the rare-entity slices they define, state-of-the-art accuracy drops by between fifteen point four and thirty-nine point nine percent.

To address those failures, the authors build a simple framework that needs no training. A vision-language model capable of reasoning searches Wikipedia iteratively and reasons over what it finds, gathering evidence dynamically. Controlled experiments then separate the two ingredients. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning does improve rare-entity accuracy, but it can hurt overall accuracy. The combination performs best.

The main test is MERLIN, a multilingual multimodal entity linking benchmark over five languages: Hindi, Indonesian, Japanese, Tamil, and Vietnamese. There, the best system improves over the state of the art by six point nine percent overall, and by up to twenty-three point three percent on rare-entity slices.

The claims are bounded by the setting. The results on the framework come from MERLIN and its five languages, and the knowledge source searched is Wikipedia. The complementarity result is about this training-free framework, built around a vision-language model that can reason, and the rarity result is about the slices that the structural metrics define. To support targeted evaluation of exactly those slices, the authors release MERLIN-Rare alongside the framework.

The title's advice, think before you link, is the finding in brief. What counts as rare is a choice of measurement, and a popularity measure hides part of the problem. Searching helps with the entities a model has barely seen, but only when the model also reasons about what the search returns. Either one alone leaves the gap open, or opens a new one.