BrickBench: Evaluating Agentic Brick Design

BrickBench. Evaluating Agentic Brick Design. By Peter Kulits and four others, from Stanford University, the Max Planck Institute for Intelligent Systems, and one other institution. Published on the eighth of October.

Designing a LEGO set from a text prompt asks two things of an agent at once. The assembly has to match what was asked for and meet design criteria. It also has to be something a person could physically build. Getting there means choosing parts from a discrete library and reasoning about local and global constraints together. A piece has to fit where it is placed, and the whole assembly has to hold together and still look like the thing in the prompt. Satisfying one of those without the other is easy. The task asks for both.

BrickBench is a benchmark for exactly this: agentic, text-conditioned LEGO-set design. Given a prompt, the agent produces an assembly, and that assembly is scored on three things. Validity, meaning whether it can be built. Alignment with the prompt. And design.

Scoring happens across three settings, which differ in scale and in which parts are available. Alongside the benchmark comes BrickAgent, an environment where coding agents can construct their designs, inspect them, and validate them. Both are released, so other agents can be tested in the same environment, against the same three settings.

Leading agents largely satisfy the requirements that can be verified, physical and semantic alike. Where they fall short is against human designs.

The checkable part of the problem, then, is mostly handled: a model can now produce a set that stands up and matches the prompt. The part that separates it from a person is the quality of the design itself.