Rate-distortion optimization for full-reference image quality metrics via stochastic Hessian estimates

Rate-distortion optimization for full-reference image quality metrics via stochastic Hessian estimates. By Samuel Fernández-Menduiña and two others, from the University of Southern California. Published on the twenty-fourth of September. Its code repository is public and real: about thirty-five source files.

A block-based video codec chooses its coding parameters by optimizing a trade-off between rate and distortion. The usual distortion is the sum of squared errors, and its appeal is structural. The error for a whole frame is simply the sum of the errors for each block, so rate-distortion optimization can treat every block independently. Full-reference image quality metrics, such as MS-SSIM or LPIPS, often agree better with the human visual system. But they cannot be used inside the coding loop. They do not break apart block by block, and they typically need the fully decoded image as input. A better yardstick exists, and the encoder cannot hold it.

The authors build on existing results in metric quadratization. A broad class of these metrics is approximated by an input-dependent quadratic distortion, called IDQD. Its quadratic form matrix comes from the Hessian of the metric, evaluated at the source video. The target metric is replaced, near the source, by a quadratic stand-in that is shaped to match it.

A quadratic form over the whole image still does not split into blocks, so the Hessian itself has to be simplified. Two approximations are proposed. The first keeps the block-diagonal part of the matrix. The second keeps only its diagonal. For both, the authors propose estimators that need nothing more than matrix-vector products with the Hessian, and those products come from automatic differentiation. The title calls these stochastic Hessian estimates.

Testing spans five metrics, on Kodak and on CLIC, inside VVC. Under the target metric, IDQD-RDO achieves bit-rate savings, measured as BD-rate, between fourteen point two percent and thirty-six point seven percent. The decoder is unchanged. The cost falls on the encoder, as an added encoding complexity of ten to thirty percent.

The contribution is a translation. A perceptual metric that a codec could not use becomes a distortion it can, because the quadratic approximation, trimmed to the block-diagonal or the diagonal, restores the one property the encoder depends on: each block can be scored on its own. The savings are paid for at the encoder and nowhere else.