Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations

Gluing Local Contexts into Global Meaning. A Sheaf-Theoretic Decomposition of Transformer Representations. By Bryce Grant and one other, from Case Western Reserve University. Published on the twelfth of October. Its code repository is worked out: about forty source files, with a readme and a licence.

Here is a question existing interpretability tools do not test. Does a representation keep the same content when only the phrasing changes? Two sentences can say the same thing in different words, and a model that has really captured the content should hold something in common across them. The tools at hand do not check for that directly.

The authors treat each paraphrase pair as a local witness: a small piece of evidence that two activations ought to agree. Those local agreements are then glued together. They build a cellular sheaf over the paraphrase graph, and the sheaf's Laplacian answers the question on a frozen model, with no training at all. The kernel of that Laplacian is read as a content subspace, the part of the representation that stays put across paraphrase. Its spectral complement is the part that moves.

Three things are drawn from that one structure. First, the content subspace itself. Second, the influence directions, which come from the complement. Third, a fragility measure. Stacking layers into a grid creates algebraic one-cycles, and from them comes a model-derived quantity the authors call harmonic mass. The same paraphrase graph yields all three.

On the content side, the kernel at twenty dimensions beats the preserved subspace of the concept eraser LEACE on held-out CounterFact fact retrieval. That holds across eight architectures, by a mean of seventeen point one percentage points. Constraining all paraphrases of a fact jointly adds a further six point five points.

On the influence side, ablating the most paraphrase-variable coordinates, a coordinate-wise proxy for the complement, changes model outputs between five point six and twenty-six point five times more than ablating coordinates of matched variability. That spans nine models, from one hundred and twenty-four million parameters up to thirteen billion. And on fragility, harmonic mass varies by nearly three orders of magnitude across six architectures, ranking them as their steering fragility does, with a Spearman correlation of zero point nine zero and an exact one-sided permutation p-value of zero point zero one four.

One graph of paraphrases, one sheaf built over it, and three readings come out: what stays fixed, what carries causal weight, and how fragile a model is to steering. None of it requires training anything on top of the model being studied.