One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

One Editor, Many Edits. A Unified Training-Free Framework for Diverse Video Editing. By Adheesh Sunil Juvekar and eight others, from the University of Illinois Urbana-Champaign. Published on the third of September. Its code repository is public but slight: seven source files.

Video editing spans a range of paradigms that are usually served by different systems. Instruction-guided editing on one side, subject-guided editing on the other. Achieving high quality on both inside a single unified framework has remained the challenging part, and that is what EditVid takes on.

The framework is training-free, which sets the constraint on everything else. Whatever holds a video together across an edit has to be assembled from mechanisms applied at inference rather than learned. Three of them are combined.

Sparse causal memory handles local coherence, keeping consecutive frames consistent with one another. Correspondence-based post-attention token injection handles long-range identity preservation, so a subject stays itself across the length of a clip and not merely across neighbouring frames. Soft latent blending handles edit locality, confining the change to where it belongs. The same three support instruction-guided and reference-guided edits alike: style transfer, attribute modification, object insertion, part-level editing, subject replacement.

On FiVE, EditVid reaches a FiVE-Acc of seventy-eight point one six, against fifty eight point nine five for the strongest training-free baseline evaluated. Results on IVEBench are competitive rather than leading. A user study reports fifty-one point eight percent overall preference for EditVid over seven competing methods.

The claim being made is unification rather than a new record. One framework, no training, and a list of edit types that would ordinarily each want a system of their own. The three mechanisms map cleanly onto the three things that break when a video is edited: local coherence, identity over time, and where the edit stops.

One Editor, Many Edits. A Unified Training-Free Framework for Diverse Video Editing. By Adheesh Sunil Juvekar and eight others, from the University of Illinois Urbana-Champaign. Published on the third of September. This is the long version: the whole paper rather than the abstract.

Image editing has become very good. Modern multimodal diffusion transformers follow instructions well, accept a reference image, and manipulate content at high fidelity across a wide range of edit types. The obvious thing to try is running one of those editors on every frame of a video. It does not work, and the ways it fails are familiar: flicker, identity drift, edits that are inconsistent from frame to frame, and changes to backgrounds nobody asked to change. So the question the paper sets itself is precise. Can the editing capability of a modern image model be extended to video without any video-specific training?

There are two established answers and each gives something up. Dedicated video editing models train on video, so they inherit temporal coherence, but they stay constrained by whatever their underlying generator can do. Training-free methods that reuse image priors keep the editability, and then have to supply temporal coupling themselves. The authors' claim is that strong image editing priors remain underexplored for video precisely because that coupling has been treated as one problem. Their finding is that it is two problems, and that they want opposite solutions.

The distinction is between short-range and long-range consistency. Consecutive frames retain strong spatial continuity, so the attention states of the immediately preceding frame are directly useful for stabilising local appearance and motion. As temporal distance and spatial displacement grow, that stops being true, and for a specific architectural reason: in these models spatial relationships inside attention are encoded by rotary positional embeddings, so reusing attention states across a large displacement makes the result sensitive to the relative geometry those embeddings encode. Hence a local-global decomposition. Locally, a sparse causal memory exposes each frame to the previous frame's key and value states, bounding the temporal context. Globally, high-confidence cycle-consistent correspondences are established between an anchor frame and later frames, and the matched visual representations are injected after attention rather than through it.

A third mechanism handles a problem the first two do not touch. Temporal consistency is not enough, since an output can be perfectly coherent and still alter backgrounds or other content the instruction never mentioned. Soft latent blending derives continuous, timestep-dependent preservation weights from the discrepancy between the source and edited trajectories, retaining source content where preservation is needed and letting the edit dominate elsewhere. On the FiVE benchmark the framework reaches an edit accuracy of seventy-eight point one six against fifty-eight point nine five for the strongest training-free baseline evaluated. A second ablation supports the long-range half of the design: filtering the correspondences, keeping only those that are high-confidence and survive a cycle-consistency check, improves robustness when the temporal changes are challenging. Correspondences that cannot be trusted are simply not used.

The results are more mixed than the headline suggests, and the paper reports them that way. On the second benchmark it takes the strongest training-free scores on total, instruction compliance and video fidelity, but ranks second overall on total once training-based methods are included. On the curated evaluation it leads five of six criteria and comes second on background consistency. The user study is the strongest single result, with a preference share above half against seven competing methods, and it rests on twenty-two participants over twenty videos. The framework also inherits whatever its frozen image editor cannot do, which is the price of not training.

The lesson is about problem decomposition rather than about video. Temporal consistency looked like one requirement, so mechanisms were built to satisfy it as one requirement, and they were pulled in two directions by a geometry that holds between neighbours and fails across a scene. Splitting it into local coherence, long-range identity and edit locality gives three mechanisms that are each simple, because each one now faces a problem with consistent structure. The sparse memory can stay sparse. The correspondence transfer can skip attention. And nothing had to be trained.