Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network

Refinement Is Inherently Editable. Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network. By Yulong Chen and five others, from City University of Hong Kong, Dongguan, City University of Hong Kong, and two other institutions. Published on the seventeenth of September. Its code repository is public and real: about twenty source files.

Text-guided image editing has to make the requested change while leaving unrelated source content alone. Diffusion-based editors depend on spatial controls, and when those are inaccurate an edit can be left incomplete, or can alter regions it should not touch. Causal autoregressive editors face an additional constraint. Their fixed decoding order limits how far they can revise earlier decisions. Once a part of the image has been committed, the editor has little room to go back if later evidence shows the choice was wrong. Both families, in different ways, struggle to decide where an edit belongs and then stay faithful to everything else.

RefineEdit is a training-free prompt-to-prompt editing framework built on a Generative Refinement Network. Its key idea is to couple locating the edit with generating the content, through global refinement of binary image codes. Because the whole image is refined, the evidence for what should be edited can be reassessed as the image evolves, rather than settled once.

An editing branch starts from an intermediate state of the source, reusing the layout that is emerging there. The two branches, source and editing, each assign probabilities to the same bits sampled from the source. The signed differences between those probabilities select which positions and which bits are editable. Selected bits follow the editing refinement. All the remaining bits copy the evolving source state. Where the two branches agree, nothing needs to change, and the source is carried through. Where they disagree, that disagreement is the evidence that a region belongs to the edit.

Two devices keep editing stable across refinement steps. Adaptive spatial freezing limits unnecessary growth of the mask. Finite bit locking keeps recently selected bits editable. None of this needs additional training, external masks, or attention control. Freezing keeps the edited area from spreading where it is not needed, and locking keeps a recent decision open long enough to be carried through.

Across nine editing categories of PIE-Bench, RefineEdit achieves the best background-preservation scores on PSNR, LPIPS, MSE, and SSIM. It also has the highest CLIP scores, both for the whole image and for the edited region, among the methods evaluated.

The title states the argument directly: refinement is inherently editable. A generator that already revisits the whole image at every step has the capacity for revision an editor needs, and RefineEdit uses it to decide, and keep re- deciding, what to change.