Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows

Argo-Bench. Evaluating Data Agents on Enterprise-Scale Workflows. By Gabriel Tomitsuka and four others, from TextQL. Published on the first of October. Its code repository is worked out: about forty source files, with a readme and a licence.

Enterprise data science and analytics work does not happen in one table. It means reasoning across dozens of them, running statistical analyses, and then acting on what comes out. The established text-to-SQL benchmarks measure only the query generation step, and audits of those benchmarks have found their answer keys frequently wrong. There is also a structural reason they look the way they do. Real enterprise warehouses are too sensitive to release, so benchmarks get built on public datasets, where a single business event fits neatly in a single table. The messiness that makes the real job hard never makes it into the test.

Argo-Bench is an evaluation framework of two hundred and ten data science and analytics tasks, and it measures more than whether a query is right. The agent files actions. It might ban fraudulent accounts, allocate courier incentive budgets, or issue back pay, and each action is graded by its consequences inside a simulator. A good answer, here, is one that changes the simulated world in the right way.

The world being simulated is a food delivery platform in New York City, built at true scale from public data, peer-reviewed industry literature and regulatory filings. It carries eighty-one million orders over the year, grounded economics, fraud patterns and marketplace incentives.

That world is exported to an ERP warehouse of two hundred and thirty-five tables and seven point five billion rows, modelled on the Oracle E-Business Suite schema. The key design decision is what the agent cannot see. The simulator's ground-truth state is withheld from the warehouse, so every task requires reconstructing the facts by navigating the warehouse before acting on them. To show none of this is impossible, every task ships with an executable reference solution that solves it using only the warehouse.

Fourteen frontier and open-weight models were evaluated. The strongest of them scores ninety-five or higher on only thirty-four point eight percent of tasks, and its average is fifty-nine point five points. Since every task is known to be solvable from the warehouse alone, that shortfall sits with the agents rather than with the tasks.

What Argo-Bench changes is the unit of success. Text-to-SQL asks whether the query matches a key. Here the agent has to find facts scattered across a large warehouse, decide what they imply, and take an action whose effect is then played out. The authors hope it pushes toward agents that understand, navigate and act within real data environments, and the gap between the best model and a perfect score shows how much room that leaves.