Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction | Yipeng Li et al. | ResearchPod

Transcript: Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction

Alex: To train an agent on company software, you need company data. And that is exactly the data nobody will hand over. This work's answer is to let an AI agent build the data itself, by using the software's own interface, so every record it creates is valid by design.

Sam: Place it for me before we go further. Whose work is this, and what kind of document?

Alex: It's a method paper from a team at SAP Labs, posted as an arXiv preprint. They call the approach Synthesis Through Simulation, and they've open-sourced the framework, ten test environments, and the generated data.

Sam: I don't build enterprise agents. Why should someone like me care?

Alex: Because anyone training or evaluating tool-calling agents, meaning language models that take actions through APIs, needs realistic databases for them to act on. Real ones contain personal data, and even the database blueprint is often confidential. This offers a way to fill a database starting from the API descriptions alone. So we'll cover why existing synthetic data falls short, the one idea that fixes it, and how far the evidence actually goes.

Sam: Synthetic tabular data is a mature field. What's wrong with just training a generator on some rows?

Alex: Two things, the authors argue. First, enterprise validity is relational, not a property of one row. A time-off request must not exceed the employee's accrued balance. That's checked against the live database at the moment of writing. Generators like CTGAN treat rows independently and have no way to enforce that.

Sam: And the second?

Alex: They need seed data to learn from. In seven of the authors' ten environments there is none, which mimics a fresh deployment. So those generators simply can't run. The other old option is scripted generation, where people hand-code the business rules. That gets validity, but it says what's allowed, not what's typical. And it has to be rewritten for every new domain.

Sam: So one camp gets realistic distributions but breaks the rules, and the other keeps the rules but needs a human per domain.

Alex: That's the tension. The fix is almost embarrassingly simple. Generate the data through the same API layer that enforces the rules. If a call violates a rule, the system rejects it and the database doesn't change. So any sequence of accepted calls leaves a valid database. They state it as a short proof by induction: start valid, and every committed write preserves validity.

Sam: That sounds true by definition. Isn't the hard part just moved somewhere else?

Alex: Partly, yes, and the authors say so. What it buys is a clean split. Validity is handed to the environment, so the agent only has to worry about realism. That's where their agent comes in, the Generalist Populator. It sees only the tool registry, meaning the list of API functions with their descriptions and inputs. No database schema, no hand-written task list, no per-domain settings.

Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction

Yipeng Li, Ashutosh Hathidara, Jane Lo, Harshavardhan Abichandani, Gunraj Singh, Atin Ghosh

Sep 24, 2026·7:12·arxiv·Original Paper

Synthesis Through Simulation (STS)Tool-calling AgentsDistributional FidelityStructural ValidityGeneralist Populator (GP)Procedure-based Synthesis

Listen

7 min

Summary

Research Question

Modern enterprise AI systems require large corpora of coherent data for training and evaluation. However, production data is often protected by strict privacy regulations and proprietary database schemas, creating a significant barrier to agent development. This paper investigates whether one can synthesize realistic, structurally valid enterprise data by leveraging the existing programmatic constraints of enterprise APIs rather than traditional statistical models.

Approach

Instead of treating data synthesis as a statistical row-generation task, the authors propose Synthesis Through Simulation (STS). In this paradigm, an LLM agent—the Generalist Populator (GP)—interacts with a simulated enterprise environment solely through its published API tools. Because these APIs already encode business logic and relational dependencies, any data generated through legitimate API calls is guaranteed to be valid. The GP agent operates through three distinct phases:

EXPLORE: The agent builds an internal model of entity relationships and tool dependencies.
PLAN: The agent generates a structured, domain-specific task description to ensure distributional coverage.
EXECUTE: The agent performs tool calls within the environment, using a long-term 'Exploration Manifest' to share heuristics across trajectories.

Findings

GP achieves 100% constraint satisfaction (structural validity) across all ten tested domains, including HR, airline reservations, and banking, without requiring access to database schemas. It consistently outperforms statistical synthesizers, which struggle with the 'cold-start' problem when no initial data exists, and even matches or exceeds the performance of schema-privileged agents that have full visibility into the underlying database. The authors demonstrate that training agents on data produced by the STS framework leads to superior performance on downstream evaluation benchmarks compared to models trained on seed data.

Why It Matters

The STS framework resolves the conflict between privacy-preserving data access and the need for realistic, relational enterprise data. By shifting the synthesis burden from the model to the API-enforcement layer, this approach provides a scalable way to generate high-fidelity datasets while respecting the architectural boundaries of enterprise systems.

Key Terms and Definitions

Synthesis Through Simulation (STS) — A data generation paradigm that uses an agentic loop against policy-enforcing APIs to guarantee structural validity by construction.
Generalist Populator (GP) — A domain-agnostic agent that synthesizes data using only a raw tool registry, without access to database schemas or prior domain-specific configurations.
Exploration Manifest — A shared memory store where the agent accumulates structural knowledge and reusable heuristics across different generation trajectories.
Validation Pass Rate (VPR) — The proportion of synthesized records that satisfy all programmatic business-rule checks in the environment, serving as the primary metric for structural validity.
Distributional Fidelity (DF) — A collection of metrics (marginal, pairwise, and cross-table) used to measure how closely the synthetic data distribution matches expected real-world reference priors.

Abstract

Tool-calling agents have become central to enterprise AI, yet training and evaluating them at scale remains severely constrained due to business and legal restrictions on enterprise systems, data, and database schemas. Tabular data synthesis offers a natural alternative, but its effectiveness is fundamentally limited by structural validity and schema availability, while procedure-based approaches yield the opposite weakness, typically lacking distributional fidelity without per-domain authoring. We introduce **Synthesis Through Simulation** (STS), a **schema--free** data synthesis paradigm in which an LLM agent generates data by executing operations against policy-enforcing APIs within simulated enterprise environments. Because data is generated through the same environment that defines what is valid, STS guarantees structural validity by construction while decoupling validity enforcement from distribution modeling, allowing each to be addressed independently. The **Generalist Populator** (GP), STS's domain-agnostic agent, addresses the remaining challenges of distributional fidelity and synthesis scalability: GP achieves **0.88** average marginal fidelity and **100\% constraint satisfaction** across all ten environments *without access to DB schemas*, while statistical synthesizers are inapplicable to seven due to necessary seed data requirements, and schema-privileged agents fail 82\% of trajectories on airline environment's tightly coupled workflows due to brittle task composition. We open-source the full framework, all ten environments, and generated datasets at https://github.com/SAP/synthesis-through-simulation.

Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction: Research Paper Podcast