When Models Edit Too Much: On the Fidelity of Minimal Code Edits

When Models Edit Too Much. On the Fidelity of Minimal Code Edits. By Tongyao Zhu and two others, from the National University of Singapore. Published on the third of September. Its code repository is worked out: about eighty source files, with a readme and a licence.

Language models are increasingly used to edit existing code, and the way they get evaluated has focused on one question: does the edit work. The argument here is that correctness alone is not enough. A useful repair should also be minimal, reviewable, and faithful to the original implementation, and nothing in a pass rate measures any of that. The behaviour left unmeasured is over-editing, the tendency of a model to rewrite code beyond what fixing the bug required.

To measure it you have to know what the minimal edit would have been, and that is the design problem. The authors solve it by construction. Starting from four hundred BigCodeBench problems, they inject controlled corruptions into the reference solutions at the level of the abstract syntax tree. Because the corruption is known, the patch that undoes it is known too, so every repair task carries a known minimal patch for an actual edit to be compared against.

With that reference in hand, two quantities become measurable alongside correctness: how far an edit exceeds the minimal one, reported as excess Levenshtein distance, and how much cognitive complexity the edit added. Measured across frontier models, over-editing is widespread, and widespread even among strong ones. A high Pass@1 can coexist with unnecessarily large edits and added complexity, which is exactly the combination a correctness-only evaluation cannot see.

A preservation instruction, simply asking the model to keep what it can, substantially reduces the behaviour. Average excess Levenshtein distance falls from zero point one nine five to zero point one three one, added cognitive complexity drops by twenty-six point six percent, and Pass@1 rises by two point three points. Those gains do not follow from a larger reasoning budget or a larger model, which rules out the easy explanation. Asked whether minimal editing can instead be learned during post-training, supervised fine-tuning overfits to the corruption patterns it has seen, while reinforcement learning gives the better out-of-domain trade-off between edit fidelity and performance retention.

The conclusion is a positioning claim. Edit fidelity is a distinct axis of code-repair quality, separate from whether the repair works, and what the paper shows is that it can be both measured and learned. A model passing the test while rewriting half the file has done something the test was never asking about.

When Models Edit Too Much. On the Fidelity of Minimal Code Edits. By Tongyao Zhu and two others, from the National University of Singapore. Published on the third of September. This is the long version: the whole paper rather than the abstract.

A function in BigCodeBench fits an exponential curve to each series and plots it. It has an off-by-one bug: a loop runs over the length of the input minus one when it should run over the whole thing. The minimal fix is a single line, and the authors verify that this one-line patch passes all five of the task's tests. Now watch what frontier models do with it. One changes three lines. One changes six. One inserts forty-six. And one deletes five lines and inserts sixty, adding input validation, dtype coercion, masking for missing values, and curve resampling. Every one of those patches passes the same five tests. None of that extra code is required by anything the task asked for.

The argument is that correctness alone was never the requirement. In real development, and particularly in brownfield work where a model edits an existing implementation rather than writing a new one, a patch also has to be reviewable and faithful to what was already there. Keeping a repair minimal reduces review burden, avoids diff churn, preserves implicit design choices, and lowers the risk of regressions outside the existing tests. The authors name the failure over-editing: a functionally correct repair that changes more code than the minimal fix required. Existing benchmarks cannot see it, because they measure whether tests pass.

Making it measurable requires knowing what the minimal patch would have been, and the framework gets that by construction rather than by judgement. Starting from four hundred BigCodeBench problems, the authors inject one or two localized corruptions into each reference solution at the level of the syntax tree, and keep the example only if the corrupted program now fails the original tests. The minimal repair is then exactly the reversal of the injected corruption, known without anyone having to decide. Three metrics follow: pass rate for functional success, a normalized token-level edit distance for how much was changed, and added cognitive complexity for structural overhead the patch introduced.

Run twenty frontier models on the identical input and median patch size ranges from two inserted lines to sixty, which locates the behaviour in the model rather than the task. The most recent releases are not uniformly more conservative than the ones they replaced. And high correctness does not protect you: one model reaches a pass rate above eight tenths while its excess edit distance is more than four times that of the most faithful model tested. Then comes the surprisingly cheap fix. Appending a single clause to the request, asking the model to keep as much of the original code as possible, shifts every one of fifty settings toward smaller edits and raises the pass rate in forty of them.

The authors' reading of that is the sentence to carry away. The prompt appears to select a different latent repair mode: the model starts treating the existing implementation as evidence to preserve, rather than as a reference to improve on. Both modes were already in there. Which one you get depends on a clause. The heaviest over-editors move furthest, nearly halving their excess distance, while the model that was already faithful barely moves at all.

The scope statement is the main one and the authors put it up front rather than at the end: this is local repair, and nothing here says a broad rewrite is wrong when a broad rewrite is what was wanted. The corruptions are synthetic and localized by design, which is what buys the known minimal patch and also what limits how far the numbers generalise. The reward design carries its own strong assumption, since an edit that grows beyond a threshold scores lower than an outright incorrect one, which is a deliberate and debatable statement about what a bad patch is. And the qualitative diagnosis stays a diagnosis: over-editing often looks like a granularity mismatch, where the model finds the bug but rewrites data flow or adds defensive checks instead of making the local repair.

What this establishes is a second axis, not a better score on an existing one. Edit fidelity is separable from correctness, it is measurable once you build a setting with a known minimal answer, and it is movable by both prompting and post-training. The uncomfortable part is how invisible it has been. Every one of those patches was green. A benchmark that asks only whether the tests pass will report a sixty-line rewrite of a one-line bug as a success, indefinitely.