SketchDirector: Complex Scene Sketching via Agentic Layout Reasoning and Detail Refinement
Yiqi Liang, Xiaodong Cun
GVC Lab, Great Bay University
Published October 30, 2026 · on the feed November 2, 2026 · ▲ 2 · arXiv 2610.41260
The abstract
Existing sketch generation methods often rely on image guidance and are largely limited to individual objects or simplified scenes, leaving text-guided complex scene sketching underexplored. We present SketchDirector, an agentic framework that mimics the human sketching workflow for automatic multi-concept scene sketching. Given a scene-level text prompt, SketchDirector first employs a large language model~(LLM) to decompose the scene into multiple draft assets, where each is represented by a set of Bézier curves. Then, an agentic reasoning loop iteratively plans and executes their spatial arrangement, guided by self-evolving memory. Finally, a detail refinement module leverages diffusion priors to enhance sketch quality while preserving geometric structure. To support future research and applications in multi-concept scene sketch generation, we further introduce the MCSketch8K dataset, comprising editable vector assets, compositional layouts, and object-level manipulation annotations. Extensive experiments show that SketchDirector outperforms state-of-the-art baselines in semantic and compositional quality while effectively preserving overall contour organization during refinement. The project page is: https://yiki77.github.io/SketchDirector/{https://yiki77.github.io/SketchDirector/}.
The Brief
1:58A short podcast summary of the paper. Written and voiced by AI.
Transcript
SketchDirector. Complex Scene Sketching via Agentic Layout Reasoning and Detail Refinement. By Yiqi Liang and one other, from Great Bay University. Published on the thirtieth of October.
Most sketch generation methods lean on image guidance, and they are largely limited to single objects or simplified scenes. Sketching a complex scene from a text description, with several concepts arranged together, has received little attention by comparison. The harder case is the one left open: no reference image to follow, many objects instead of one, and a layout that has to make sense as a whole.
SketchDirector is an agentic framework built to mimic the way a person sketches: rough out the pieces, arrange them, then refine. Given a scene-level text prompt, it produces a multi-concept scene sketch automatically. Each of those three human steps becomes a stage of its own, handled by a different component.
The work runs in three stages. First, a large language model decomposes the scene into several draft assets, each one represented as a set of Bézier curves. Next, an agentic reasoning loop plans and executes the spatial arrangement of those assets, iterating as it goes, and guided by a memory that evolves itself along the way. Last, a detail refinement module draws on diffusion priors to raise the quality of the sketch while preserving the geometric structure already in place.
Alongside the framework, the authors introduce a dataset, MCSketch8K, to support research and applications in multi-concept scene sketch generation. It contains editable vector assets, compositional layouts, and annotations for manipulating individual objects.
In extensive experiments, SketchDirector outperforms state-of-the-art baselines on semantic quality and on compositional quality. It also preserves the overall organization of contours during refinement, so the detail pass improves the sketch without undoing its layout.
What carries the design is an order of operations borrowed from people: decompose, arrange, refine. Layout gets a reasoning loop of its own before any detail is added, and the refinement that follows is held to the structure that loop produced.
More with an account
Listen to every day’s papers
Daily playlists, deep dives, quests for the papers, and quizzes on their key terms.
Join free