Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching

Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching. By Chuangtao Ma and three others. Published on the sixth of November.

Schema matching is the job of deciding which fields in one data schema correspond to which fields in another. The traditional approach compares similarity, and on complex, domain-specific mappings that approach fails to resolve semantic ambiguities and conflicts. The reason given is a lack of knowledge: both commonsense knowledge and the specialised knowledge of the domain. Large language models look like a natural fix, but without sufficient background knowledge they tend to hallucinate, and a hallucinated match makes schema matching harder rather than easier. Two families of method, then, each missing the same ingredient.

KG-RAG4SM supplies that ingredient from outside. It is retrieval-augmented generation for schema matching, where the material retrieved comes from large external knowledge graphs. Rather than asking the model to know everything, the method finds the most relevant subgraphs and hands them over as context. All of this works without re-training the model.

Retrieval is where most of the design sits. The method integrates several retrieval mechanisms: one based on the language model itself, one based on vectors, one based on graph traversal, and hybrid techniques that combine them. Advanced reranking schemes then sort what was found, so that the subgraphs passed along are the most relevant ones. A further piece addresses the case where retrieval falls short. An instruction-tuning knowledge exclusion component ensures truthful performance in end-to-end matching, even when the knowledge graph context is of suboptimal relevance.

The comparisons run against two kinds of state of the art. Against methods built on large language models, with Jellyfish-8B as an example, on the MIMIC dataset, precision improves by thirty-nine point seven nine percent and F1 score by thirty point five six percent.

Against methods built on pre-trained language models, with SMAT as the example, on the Synthea dataset, precision improves by sixty-nine point two percent and F1 score by twenty-one point nine nine percent. Beyond those figures, the method is reported to scale effectively across knowledge graphs, from commonsense ones to domain-specific ones. An empirical analysis, along with case studies from real-world scenarios, shows the hallucination problem in schema matching being well mitigated.

Knowledge is the missing piece in both of the failures described at the start, and the answer here is to fetch it rather than to train it in. Similarity methods lacked the background to resolve ambiguity, and language models lacked it enough to invent matches. Retrieving the right subgraph, and excluding it gracefully when it is not right, addresses both at once without touching the model's weights.