AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation
Yuta Oshima, Ku Onoda, Yusuke Iwasawa, Masahiro Suzuki, Yutaka Matsuo, Hiroki Furuta
The University of Tokyo
Published September 28, 2026 · on the feed September 30, 2026 · ▲ 17 · arXiv 2609.35530
The abstract
Recent image generation models can take multiple reference images as input and combine them into a new image. However, multi-reference image generation remains challenging: models may omit or duplicate subjects from the references, or produce images in which multiple subjects appear unnaturally pasted. Recent work has proposed image generation agents that combine image generation models, reasoning models, and a harness, which is an executable program that specifies how reference images are interpreted, how generation is performed, how outputs are diagnosed, and how the final image is selected. In multi-reference generation, however, references play different roles and outputs must satisfy many criteria at once, such as fidelity to each reference and the naturalness of the whole image, so many parts of the harness could be improved, from how references are processed to how outputs are diagnosed. This makes it hard to predict which changes will improve performance and by how much, and good harnesses difficult to design by hand; indeed, human-written harnesses vary widely in performance. We therefore propose AutoRef, which optimizes the harness automatically while keeping both models frozen: a coding agent iteratively rewrites the harness code. AutoRef separates the tasks whose feedback informs proposals from the tasks used to select candidates, and continues the search from a beam of the top-ranked harnesses on the selection tasks. Using this procedure, we discover AutoRef-Harness, which improves the open-weight FLUX.2 [klein] 4B from 5.72 to 7.37 on held-out four-reference tasks of the MultiBanana benchmark, matching or exceeding proprietary models including Nano Banana Pro and GPT-Image-1.5. Without re-optimization, the same harness also improves results when the generator, number of references, benchmark, evaluator, or reasoning model differs from those used in the search.
The Brief
3:04A short podcast summary of the paper. Written and voiced by AI.
Transcript
AutoRef. Harness Optimization for Agentic Multi-Reference Image Generation. By Yuta Oshima and five others, from the University of Tokyo. Published on the twenty-eighth of September. Its code repository is public and real: about twenty-five source files.
Recent image generators can take several reference images and combine them into one new picture. Doing that well is still hard. A model may leave out a subject from the references, or duplicate one, or produce an image where the subjects look unnaturally pasted together.
One response has been image generation agents. They combine an image generation model, a reasoning model, and a harness: an executable program that specifies how the references are interpreted, how generation runs, how outputs are diagnosed, and how the final image is chosen. With multiple references, the harness has a lot to get right. References play different roles, and an output has to satisfy many criteria at once, faithfulness to each reference as well as naturalness of the whole. Almost any part of the harness could be improved, from reference processing to diagnosis, and it is hard to predict which change will help, or by how much. Designing a good harness by hand is difficult, and human-written harnesses vary widely in how well they perform.
AutoRef optimizes the harness automatically. Both models stay frozen: the image generator and the reasoning model are left exactly as they were. What changes is the harness code, which a coding agent rewrites iteratively. If the space of possible harnesses is too large and too unpredictable for a person to search by hand, the proposal is to let an agent search it, one rewrite at a time, and keep what scores well.
Two choices shape the search. First, AutoRef keeps apart the tasks whose feedback informs new proposals and the tasks used to select among candidates. The examples that suggest a change are not the examples that decide whether it stays. Second, rather than following one line of edits, the search continues from a beam of the top-ranked harnesses on the selection tasks. Several promising versions are kept alive at once, and each round of rewriting builds on the current leaders rather than on a single candidate.
The search produced AutoRef-Harness. On held-out four-reference tasks from the MultiBanana benchmark, it lifts the open-weight FLUX.2 klein 4B model from five point seven two to seven point three seven. That matches or exceeds proprietary models, including Nano Banana Pro and GPT-Image-1.5. The tasks were held out, so the harness was not tuned on the examples it is scored on here.
The harness also travels. Without being re-optimized, it still improves results when the generator, the number of references, the benchmark, the evaluator, or the reasoning model is different from the ones used during the search. That list matters, because a harness found by search might have been fitted to the exact setup it was searched in. The reported transfer covers five different kinds of substitution, and the harness helped under each of them.
Ask where the improvement came from and the answer is neither model. Both were frozen throughout. Everything that changed lives in the program between them, rewritten by a coding agent, and that alone was enough for an open-weight generator to match proprietary systems on this benchmark.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Four pictures in, one picture out. Something always goes missing. Here is the wall.
You generate images that combine several reference pictures: a person, an outfit, a background, a style. The generator often drops a subject, duplicates one, or makes everyone look pasted in.
The road ahead
- 1 · The problem ~6 min Combining several reference pictures into one image often fails, and the program wrapped around the generator is hard to design well by hand.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Keep both models frozen and let a coding agent rewrite the harness code, judging rewrites on separate tasks and keeping a beam of leaders.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Each round, a coding agent writes new harnesses, they are ranked on hidden selection tasks, the top two carry on, and the winner is tested on held-out tasks.
- 6 · The experiment ~6 min You’ll guess: With the harness found by AutoRef, how did the open-weight FLUX.2 [klein] 4B compare with proprietary models such as Nano Banana Pro on held-out four-reference tasks?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.