FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance

FoMo. Forking Moment in Generative Trajectory as a Perceptual Distance. By Jaihyun Lew and four others, from Seoul National University. Published on the twenty-second of September. Its code repository is public and real: fourteen source files.

Reference-based image quality assessment metrics try to capture how people perceive the distance between two images. To learn how the human visual system behaves, recent metrics of this kind lean heavily on human-annotated data, and both of the usual forms of annotation have a cost.

One form is pointwise scoring based on a mean opinion score, where each image gets a scalar quality value. That format is preferable for annotation, but it is prohibitively expensive to collect at scale, and it is known to be noisy, because human judgments are inconsistent. The alternative is pairwise labels from a two-alternative forced choice. These have become popular because they are reliable and efficient, but they only record relative comparisons: this pair versus that one. Neither gives cheap, dependable, absolute distances. The pointwise format has the shape one wants but the wrong price and the wrong noise. The pairwise format has the right price and reliability but the wrong shape.

The authors propose to take humans out of labelling altogether, with a fully automated pipeline that generates pointwise perceptual distance labels for image pairs. The source of those labels is the generative dynamics of diffusion models. In a diffusion model the coarse structure of an image is generated in the early timesteps, and the fine details in the later ones. That ordering, from coarse to fine, can serve as a proxy for perceptual distance. The model already separates what an image broadly is from how it is finished, and it does so in time, one step after another. The pipeline reads that timeline as a ruler.

Picture two images sharing one generation process and then forking apart at some moment. If they fork early, they share only coarse structure, and they are perceptually far apart. If they fork late, they differ only in fine detail. The moment of the fork therefore acts as a measure of how different the two images look. The authors demonstrate that the diffusion trajectory aligns well with the human visual system, and they name this forking moment FoMo. It becomes a reference-grounded distance label, used to supervise the training of a reference-based quality metric.

The pointwise nature of these labels matters. Because every label is a distance rather than a preference, any image pair can be compared with any other, which the authors describe as universal comparison. That in turn enables a training objective that carries more information than relative comparisons alone.

Extensive experiments across diverse backbone architectures confirm the pipeline's effectiveness. Metrics trained on its labels outperform those trained on human-annotated datasets, across multiple benchmarks. The comparison sets the pipeline's own question directly: automatically generated labels against the human annotation they were designed to replace, with no person in the loop on the generated side.

In one sentence, the paper swaps the teacher. Human judgment stops being the source of a perceptual metric's labels, the internal timeline of a generative model takes its place, and the reported result is that labels made without a single human annotation beat datasets that needed many.