Critical Percolation as a Synthetic Data Model for Interpretability

Mechanistic interpretability aims to reverse-engineer neural networks, but progress is often hindered by the lack of synthetic datasets that accurately reflect the complex, hierarchical, and multi-scale structure of natural data. The authors seek to bridge this gap by developing a principled, analytically tractable synthetic data model that mimics the statistical properties of real-world information.

The researchers propose a model based on critical mean-field percolation theory. In this framework, data points are organized into sparse, fractal clusters embedded in high-dimensional space. The model is defined by three key properties: mean-field behavior (assuming high dimensionality), criticality (operating at the phase transition point), and a taxonomic hierarchy. To make this model computationally feasible, the authors leverage a mathematical mapping between percolation clusters, random labeled trees, and the additive coalescent process. They introduce the "cyclic coalescent" algorithm, which uses a union-find data structure to jointly sample a random tree and its hierarchical latent decomposition in almost linear time, O(n α(n)).

The percolation model successfully reproduces several hallmarks of natural data, including sparsity, self-similarity, power-law cluster size distributions, and low intrinsic dimensionality. By training neural networks on this synthetic data, the authors demonstrate that the model's ground-truth hierarchical latent variables can be recovered via linear probing. This confirms that the synthetic data provides a rigorous testbed for evaluating interpretability methods, such as sparse autoencoders, by providing a clear ground truth for feature splitting and hierarchical organization.