Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness

Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness. By Aneesh Komanduri and three others, from the University of Colorado Anschutz Medical Campus, the University of Wisconsin–Madison, and two other institutions. Published on the fifteenth of October. No code repository is linked to it.

Large language models are increasingly used in critical domains. Because they are susceptible to hallucination, those domains need uncertainty estimates that are both reliable and interpretable. The conventional methods judge uncertainty from the output. This work looks underneath it instead, at the machinery inside the model that produces a response, and asks how easily that machinery can be moved.

The authors introduce mechanistic uncertainty: uncertainty characterized by how stable the internal mechanisms supporting a response are. To measure it, they propose the causal robustness margin, defined as the minimum internal intervention needed to make the model's output deviate.

The margin is operationalized with Jacobian Lens concept steering vectors, known as Jlens, which supply the internal interventions. Read plainly, a small margin means a small push inside the model changes what it says. A large margin means the answer holds under a larger push.

Across three open-source models and five question-answering datasets, incorrect and uncertain responses show smaller margins, which indicates greater sensitivity to internal interventions. The margin also complements conventional output-based uncertainty methods. Last, in a controlled knowledge injection experiment, facts that were correctly learned show increased margins after training. Wrong answers sit on fragile mechanisms, and learning a fact makes its mechanism sturdier.

That final experiment ties the measure to learning itself. If acquiring a fact makes the mechanism behind it harder to disturb, then robustness to intervention is an uncertainty signal grounded in the mechanism, not only in the surface of the output. That is the case the authors make for causal robustness.