Energy-Tweedie: Score meets Score, Energy meets Energy

Energy-Tweedie. Score meets Score, Energy meets Energy. By Andrej Leban, from the University of Michigan. Published on the seventh of November.

Denoising and score estimation have long been tied together by Tweedie's formula. Under Gaussian noise, the formula relates the posterior mean of the clean signal to the Stein score of the noisy marginal distribution. That link belongs to a familiar package: Gaussian noise, posterior means, the squared loss, and Tweedie's formula, all fitting together. The question here is what happens to that package when the noise is no longer Gaussian.

The work extends the perspective to a broad class of Gibbs noise distributions, which are energy-based, with the generalized Gaussian family serving as an example. The central result is the Energy-Tweedie identity. When the denoising posterior is viewed through the lens of scoring rules, the path derivative of a kernel scoring rule, defined by the noise potential, recovers the Stein score of the noisy marginal. Each noise potential therefore gets its own Tweedie-style relation.

The shape of the argument is a lift from points to distributions. In the classical picture, the object of interest is a single summary of the posterior, its mean, scored by squared loss. In the new picture, the object is the full posterior law, scored by a kernel scoring rule whose kernel comes from the noise itself. Each element of the old correspondence has a counterpart. Gaussian noise becomes Gibbs noise. Posterior means become full posterior laws. Squared loss becomes a kernel scoring rule. Tweedie's formula becomes the Energy-Tweedie identity. The score of the noisy marginal is what stays fixed on both sides, and the path derivative is what connects the scoring rule to it.

The noise must belong to the Gibbs, energy-based class, described through a noise potential, and the kernel scoring rule is the one that potential defines. The generalized Gaussian family is the worked example the abstract names, rather than the boundary of the class.

Three consequences follow from the identity. It gives a route from posterior samples to the score, so that score estimation can go through samples rather than means. It yields a principled criterion for estimating unknown noise parameters. And it enables diffusion-style sampling along paths, chosen by the user, through the space of noise parameters. Together these supply a score-based perspective on recent generative methods trained with scoring rules: a familiar Gaussian fact, generalised until it covers a whole family of noises at once.