EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?

EngiWorld. What Can Frontier Agents Deliver in Professional Engineering Environments?. By Hongcheng Gao and twenty-one others, from Tsinghua University, Zhiman, and nine other institutions. Published on the twenty-ninth of September. Its code repository is worked out: about two thousand five hundred source files, with a readme and a licence.

Autonomous agents have made rapid progress at general-purpose computer use. Professional industrial engineering is a different matter, and reliable automation of it remains out of reach. Engineering workflows demand reasoning over geometric and physical constraints, and over dependencies that must be preserved as work moves across software and across design stages. Measuring an agent here means following the work through that whole chain, and judging what the agent actually produced along the way.

EngiWorld is presented as the first benchmark structured around the complete design loop. It holds one thousand three hundred and one expert-curated tasks. They span six engineering domains: CAD, CAE, CAM, BIM, EDA, and 3D visualization. They run on twenty-six professional software platforms, through both graphical and command-line interfaces, and they come in six task types, ranging from simply selecting the right software up to open-ended tasks.

Scoring is artifact-centric. A unified suite of domain verifiers programmatically checks what the agent makes, and it checks three properties: geometric validity, physical feasibility, and compliance with rules. Those checks apply not only to the final artifact but to intermediate ones as well, so a design that goes wrong halfway is caught where it went wrong. Quantitative design tasks are scored continuously, by how far the result attains its specification, rather than as a binary of success or failure. Partial progress toward a target therefore registers as partial credit.

Seven frontier models were evaluated, and the result is a substantial capability gap. The strongest model reaches an EngiScore of only forty-four point three. Tasks that cross more than one piece of software are harder still: just three point six percent of multi-software attempts succeed.

What EngiWorld offers is a foundation for measuring progress toward agents that operate professional engineering software end to end. On the evidence of these numbers, that end is still some distance off, and the lowest figure of all belongs to the tasks that carry work from one program into the next.