Belief-Trajectory Energy: Measuring the Path to a Prediction

Belief-Trajectory Energy. Measuring the Path to a Prediction. By Jiahao Ying and nine others, from Fudan University, the University of Science and Technology of China, and one other institution. Published on the fourth of October. No code repository is linked to it.

A large language model does not arrive at its prediction in one step. It revises that prediction progressively, layer by layer, as an input passes through the Transformer. Yet the usual practice is to look only at the final output. Whatever happened along the way, the whole trajectory through which the answer was formed, gets discarded. The authors treat that discarded path as information that has simply been thrown out.

Their proposal is Belief-Trajectory Energy, or BTE. It is a measure that is grounded in the model itself, and it characterizes an input by the layerwise predictive revisions that input induces. Put differently, the input is described not by what it is on the surface, but by how much, and in what way, it makes the model change its mind as it moves through depth.

To make revisions at different layers comparable, intermediate states are mapped into a shared predictive space. Within that space, BTE gives a principled measure of belief change, and the measure can be summarized in two forms: a single scalar, or a structured depth profile that keeps the shape of the revision across layers. There is a theoretical grounding as well. Local BTE is shown to correspond to predictive revision under the Fisher-Rao geometry. And the sequence of revisions is shown to capture information beyond the simple change from the initial belief to the final one, which is the formal case for keeping the path rather than only its endpoints.

Empirically the two forms do different jobs. Scalar BTE serves as a model-relative signal of difficulty across diverse reasoning tasks. The richer representations support human versus LLM review detection, and fine-grained attribution of which generator produced a text. Those reach up to zero point nine nine eight macro-AUROC, and ninety-five point six percent accuracy on eight-way attribution.

Further analysis looks at where the signal comes from. BTE develops throughout pretraining, and targeted training reshapes it selectively. The authors read this as evidence that the measurement reflects what the scoring model has actually learned, rather than some fixed property of the text.

The broader suggestion is a change of role. A trained model is usually the thing being measured. Here it becomes the instrument, and the data it processes becomes the thing under measurement, read off from the path its beliefs take on the way to an answer. The authors present belief trajectories as a principled, model-grounded signal, and point to an accompanying project site with more demonstrations.