Adaptive Fused Prior Transfer for Controllable Generative Image Compression

Adaptive Fused Prior Transfer for Controllable Generative Image Compression. By Yifei Pei and two others, from Santa Clara University. Published on the twenty-eighth of September. Its code repository is worked out: about seventy source files, with a readme and a licence.

Learned image compression already reaches competitive rate-distortion performance. The trouble begins at very low bitrates, where the transmitted representation simply cannot hold on to fine textures and local structures. Two lines of work respond to that. Perceptual and generative codecs fill in the missing detail, synthesizing it from reconstruction priors. Controllable codecs let one model cover different bitrates and different reconstruction preferences. Where the two meet, in codebook-based controllable designs, the reconstruction prior generally comes from a single codebook, and that is the limitation this work starts from.

AFP-GIC, short for Adaptive Fused Prior Transfer for Controllable Generative Image Compression, swaps the single-codebook prior for an adaptive fused prior. That prior is not learned from scratch. It is transferred from a frozen, pretrained AdaCode model, and the codec built around it stays controllable.

The fused prior plays a part on both sides of the channel. On the encoder side, fused-prior features guide how the latent is formed. On the decoder side, the fused prior is never received. Instead the decoder predicts a compatible fused prior from two things it does have: the compressed representation, and the control variables that were selected. Reconstruction is then guided by that predicted prior, so the benefit of the prior arrives without spending any bits on transmitting it.

A motivating analysis supports the design in two ways. Better alignment of the decoder-side fused prior tightens an upper bound on reconstruction error. And the fused-prior family contains the single-codebook choices as special cases, so the older designs sit inside the new one rather than beside it.

The comparison point under a unified benchmark is DC-VIC. Against it, AFP-GIC has eighteen point one percent lower decoder latency. It also uses thirty-one point one million fewer inference parameters, a reduction of twenty point five percent. On quality, experiments on Kodak, CLIC2020, and DIV2K show competitive PSNR and SSIM. The clearest perceptual gains appear in NIQE scores and in visual comparisons at very low bitrates, which is the regime the work set out to address.

Put plainly, the codec borrows a richer prior than one codebook can offer, and borrows it in a way that costs nothing on the wire. The encoder uses the prior, the decoder reconstructs its own estimate of it, and the analysis says a closer estimate means a tighter error bound. Lower latency and fewer parameters than DC-VIC come along with it.