EditHero: A Benchmark for Long-Horizon Part-Level 3D Editing and Vibe Modeling

EditHero. A Benchmark for Long-Horizon Part-Level 3D Editing and Vibe Modeling. By Ruihan Yu and eleven others, from Alaya Lab, the University of Tokyo, and two other institutions. Published on the first of October. Its code repository is a placeholder: a readme, and nothing to run yet.

3D editing methods are usually tested on a single edit. An asset, though, is built through a long sequence of revisions, and each revision has to implement the requested change while leaving everything else unchanged. A single-edit test cannot show whether a method keeps that discipline across many steps in a row. Every edit in a sequence is a fresh chance either to miss the request or to disturb something that should have been left alone, and a single-edit test sees only one such chance.

EditHero is, to the authors' knowledge, the first benchmark for long-horizon, part-level 3D editing. Instructions are given in natural language, with target images for both geometry and texture. What gets checked is the double demand from the start: whether each edit implements the requested change, and whether the parts that should stay fixed actually do.

A deterministic assembly engine produces the exact target after every edit, so there is always a precise answer to compare against, however long the sequence runs. Every sequence is also reviewed by hand. Edits work at the level of parts, so a single instruction targets one piece of the object, and everything outside that piece has a known, exact form it should still have afterwards.

The benchmark is used to compare two opposite approaches to 3D editing. Non-agentic methods operate top down. They regenerate the object from a learned 3D representation, and infer what to keep. LLM and VLM agents operate bottom up. They edit through code that inspects the mesh and rewrites only the parts the instructions require. One approach reconstructs the whole and has to work out what to preserve. The other touches only what it was told to touch.

The non-agentic methods often miss the requested change, and disturb regions that should stay fixed. Most of the LLMs follow instructions more closely, and all of them preserve the unedited parts better. Their cost is time: each of their edits takes minutes.

The authors will release the engine and the edit sequences. What the comparison leaves on the table is a trade. Regenerating from a learned representation struggles to hold still what should not change, while editing through code keeps the rest intact and follows instructions more closely, at the price of minutes per edit.