LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures

LMBuild. Evaluating LLM Agents for Generating Buildable and Functional Structures. By Jiateng Liu and eleven others, from the University of Illinois Urbana-Champaign. Published on the third of October. Its code repository is worked out: about one hundred source files, with a readme and a licence.

Agents built on large language models are getting increasingly capable of generating complex 3D structures, and the authors see in that a potential to reshape how objects are designed and then realized in the physical world. But elegant geometry and a working object are different things. A chair can look right on screen and still be impossible to assemble, or fall over, or fail at the one job a chair has. Existing evaluations largely score geometric quality and overlook physical realizability. They measure whether the shape is good, not whether the thing could be built and would perform its intended function.

LMBuild is a benchmark aimed at that second question: can an LLM agent generate structures that are buildable and functional. To make the question answerable, it changes what a generated object is. Rather than a single surface or mesh, an object is represented as an assembled structure, made of a part decomposition, the joints between parts, the materials, and the sequence in which everything goes together. Evaluation then covers four things: structural soundness, functional affordance, which is whether the object offers the use it is meant to offer, design quality, and physical realization.

The framework has three pieces, built for reproducible evaluation. The first is an interactive environment where agents use tools to retrieve existing components, create new ones, and place them, constructing an object step by step. The second is a curated benchmark that repurposes established CAD datasets and augments them with knowledge drawn from Wikipedia. The third is the evaluation framework covering those four dimensions, from soundness through to physical realization.

The authors evaluate thirty systems, and three findings stand out. For frontier closed-source models, soundness and alignment are no longer the primary bottlenecks. What remains substantially harder is functional affordance and physical operability: the models can produce something that holds together and matches the request, yet struggle to produce something that works.

The second finding concerns how models build. Stronger models more effectively create new components, while weaker models tend to lean on retrieval, assembling from parts that already exist. The third is about instructions. Giving the agent functional specifications, a statement of what the object has to do, substantially improves part completeness, kinematics, and physical operability.

Read together, the three findings point the same way. Generating structures for the real world asks for deeper reasoning about functional affordances and mechanics, and for the ability to design and create novel components rather than fetch familiar ones. Geometry has become the comparatively easy part. The authors offer LMBuild as a foundation for measuring progress toward agents whose output can be built and used, and as an incentive for research in that direction.