Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures

Score-based diffusion models have achieved remarkable success in generative tasks, but existing theoretical foundations often rely on restrictive assumptions—such as Lipschitz continuity or smooth manifold supports—that are frequently violated by real-world data. This paper addresses whether diffusion models can theoretically handle irregular, non-smooth data structures, such as those with sharp boundaries, singularities, or disjoint clusters, without suffering from the curse of dimensionality.

The authors establish a universal score approximation theorem for any distribution supported on a compact set of upper Minkowski dimension d. They move away from differential geometry-based approaches, instead utilizing an analytics-heavy method centered on the smoothing properties of Gaussian kernels. By decomposing the target distribution into a discrete mixture of local components, the authors prove that the score function can be approximated by a ReLU network. This approach avoids the need for smoothness assumptions on the underlying density, instead introducing a weak regularity condition that is satisfied by a broad class of distributions, including fractal-like structures.

The study demonstrates that the score function can be approximated with a mean squared error of epsilon using a ReLU network whose size grows polynomially with the ambient dimension n and exponentially only with the intrinsic dimension d. Specifically, the network complexity is bounded by O(n^(3d/4) * (1/epsilon)^(d/2)). This result confirms that diffusion models are theoretically capable of adapting to irregular, non-smooth data structures, providing a formal explanation for their empirical success in high-dimensional generative tasks.

This work bridges the gap between the empirical performance of diffusion models and their theoretical grounding. By relaxing the requirement for smooth data distributions, the authors provide a more robust framework that aligns with the reality of perceptual data. The findings suggest that the effectiveness of diffusion models is not merely a result of heuristic design but is rooted in their fundamental ability to approximate complex, low-dimensional structures efficiently.