DATAFOUNDRY: EVOLVING DATA PREPARATORS VIA RECURSIVE SELF-IMPROVEMENT

Large language models (LLMs) often struggle with domain adaptation because data-quality issues are typically addressed only after the data has been generated, using post-hoc filtering. This paper asks whether it is possible to shift quality control upstream by recursively evolving the data-preparation pipeline itself, rather than just filtering its outputs.

The authors introduce DataFoundry, a framework that treats a data preparator as an evolvable runtime specification. It uses a Skills-as-Modules architecture where a central controller manages three key functions: compiling the specification into an executable runtime, diagnosing data quality on small pilot sets using domain-specific criteria, and translating that feedback into adapters that revise the pipeline components. By isolating domain-specific changes in these adapters while preserving stable interfaces, the system recursively improves the data-construction process before full-scale production.

DataFoundry consistently outperforms existing data-preparation baselines across four specialized domains: mathematics, finance, law, and medicine. Experiments show that the framework's improvements are not tied to a specific backbone model, as the generated datasets provide transferable utility across different model families. The authors also demonstrate that while recursive self-improvement significantly enhances data quality, excessive iterations can lead to system collapse, suggesting that the evolution process must be bounded.

This work addresses a fundamental mismatch in current data-centric AI: the disconnect between where data-quality issues originate (the construction process) and where they are addressed (post-hoc filtering). By providing a systematic way to evolve the construction pipeline, DataFoundry offers a more efficient and effective path for domain adaptation, reducing the reliance on massive, noisy datasets and manual filtering.