CDFM: Towards a General-Purpose Causal Discovery Foundation Model

Research Context and Motivation

Traditional causal discovery has long been a fragmented, manual process. Researchers typically select specific algorithms—such as PC, FCI, or LiNGAM—based on rigid assumptions about the data, such as linearity, non-Gaussianity, or the absence of latent confounders. As data heterogeneity and volume increase, this test-driven paradigm becomes a bottleneck, as it requires practitioners to repeatedly verify assumptions and manually tune models for every new dataset. The authors propose a paradigm shift: moving away from specialized, assumption-bound algorithms toward a unified Causal Discovery Foundation Model (CDFM) capable of zero-shot structural inference.

The Variational Framework

To ground the foundation model in theory, the authors analyze the limits of causal identifiability. They demonstrate that identifying causal direction is impossible without prior constraints on the data-generating mechanism. By treating unknown causal mechanisms as latent variables, they formulate a variational framework that decomposes the intractable marginal likelihood of a causal graph into three tractable modules: mechanism inference, data reconstruction, and graph inference. This theoretical decomposition serves as the blueprint for the CDFM architecture, ensuring that the model remains consistent with the mathematical requirements for causal identifiability.

Architecture and Pretraining

CDFM is designed to implement this variational framework through a deep neural architecture. It processes observational data using alternating row-column attention blocks to capture complex dependencies and statistical footprints. The model is pretrained on a massive, diverse synthetic dataset generated from a wide range of identifiable causal mechanisms. This pretraining allows the model to internalize statistical asymmetries—such as those found in additive noise or post-nonlinear models—enabling it to infer causal structures directly from new, unseen datasets without requiring manual algorithm selection or assumption checking.

CDFM represents a significant step toward automating scientific discovery. By shifting the burden from manual algorithm selection to large-scale, diverse pretraining, the model provides a scalable way to handle heterogeneous real-world data. This approach creates a self-reinforcing loop: as the model is applied to more real-world domains, the insights gained can be used to refine the underlying causal knowledge base, further expanding the diversity of synthetic data and improving the model's generalization capabilities.