Lipschitz-Controlled Flow Matching for Robust Image Restoration

Lipschitz-Controlled Flow Matching for Robust Image Restoration. By Sedjro Salomon Hotegni and three others, from Paderborn University and TU Dortmund University. Published on the tenth of November.

Flow-matching models can produce high-quality image restorations. The concern the authors raise is about what the learned dynamics do to small perturbations of the input: those dynamics may amplify them. A small disturbance enters the observation and comes out the far end of the integration larger than it went in. A reconstruction that moves a lot when its input barely moves is an unstable one, however good it looks on a clean input.

LipsFlower is their answer, aimed squarely at reducing that amplification. The analysis comes first. They derive stability bounds for two formulations. One is Flow-from-Y, which initializes the ordinary differential equation from a noise-perturbed observation. The other is observation-conditioned posterior sampling, which keeps conditioning the velocity field on the observation throughout integration. Two different ways of letting the observation into the process, each with its own bound on how much amplification it permits. The bounds from that analysis are what the design is then built on, rather than the design coming first and the theory being fitted around it afterwards.

The vector field inside LipsFlower is a Lipschitz-aware hourglass diffusion transformer called LipHDiT. It incorporates stabilized approximate spectral normalization, bounded modulation, and convex residual mixing. On top of the architecture there is a training term: adversarial finite-radius gain regularization, applied at sampled interpolation states, which penalizes excessive sensitivity to the state. For conditional sampling it penalizes excessive sensitivity to the observation as well.

Both formulations are tested on blind face restoration, and the measurement has two sides: adversarial robustness, and the perceptual quality of the reconstructions. LipsFlower reduces perturbation amplification while keeping restoration quality competitive with the baselines evaluated.

Robustness and fidelity are easy to trade against each other, and this result reads as a claim that they need not be traded hard. Bound how much the dynamics can amplify, and the authors suggest the stability of generative image restoration improves without the restorations themselves getting worse.