Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction

Synthesis Through Simulation. Generating Coherent Enterprise Data via Scalable Agent-System Interaction. By Yipeng Li and five others, from SAP Labs. Published on the twenty-fourth of September.

Tool-calling agents have become central to enterprise AI, but training and evaluating them at scale is severely constrained. Business and legal restrictions limit access to enterprise systems, their data, and their database schemas. The two existing alternatives fail in opposite directions. Tabular data synthesis is a natural substitute, but it is fundamentally limited by structural validity and by whether a schema is available. Procedure-based approaches have the reverse weakness: they typically lack distributional fidelity unless someone authors them domain by domain.

Synthesis Through Simulation, or STS, is a schema-free way of producing data. An LLM agent generates the data by executing operations against APIs that enforce policy, inside simulated enterprise environments. Since the data comes out of the same environment that defines what counts as valid, structural validity is guaranteed by construction. That also separates enforcing validity from modelling the distribution, so each can be handled on its own.

The remaining problems, distributional fidelity and scaling up synthesis, fall to the Generalist Populator, or GP, the domain-agnostic agent within STS. It operates without access to database schemas. Validity is the environment's job, so the agent's job narrows to the other two: making the data look like the real distribution, and producing enough of it across many environments without per-domain authoring.

Across all ten environments, GP achieves an average marginal fidelity of zero point eight eight, and one hundred percent constraint satisfaction, without seeing a schema. The baselines struggle in two different ways. Statistical synthesizers cannot be applied to seven of the ten environments, because they need seed data. Agents given privileged access to schemas fail eighty-two percent of trajectories on the airline environment, whose workflows are tightly coupled, and the stated cause is brittle task composition. The full framework, all ten environments and the generated datasets are open-sourced, under the SAP organisation on GitHub.

The central move is to let validity come from the system rather than from a schema. An agent that can only act through rule-enforcing interfaces cannot produce invalid records, which leaves fidelity as the one problem to solve, and it frees the approach from needing the very schemas that enterprises are least able to share.