A MATHEMATICAL INTRODUCTION TO DIFFUSION MODELS

These lecture notes offer a systematic, proof-based introduction to the mathematics of diffusion models. The author traces the evolution of generative sampling from classical Langevin dynamics to modern score-based diffusion models. The text is structured into five movements, covering the sampling toolkit, continuous-time score-based diffusion, discretization into implementable samplers (such as DDPM), discrete diffusion on finite state spaces, and inference-time steering techniques like guidance and reinforcement learning.

The central strategy for generative modeling in this framework is indirect: rather than sampling a target distribution directly, one constructs a random process that is easy to simulate and whose distribution gradually drifts toward the target. The notes emphasize tracking the evolution of the entire probability density rather than individual trajectories. This approach allows for the rigorous analysis of convergence using tools like the Fokker–Planck equation, entropy dissipation, and functional inequalities such as the log-Sobolev inequality.

A core contribution is the derivation of the reverse-time SDE and the probability-flow ODE. The paper demonstrates that the score function—the gradient of the log-density of the noised data—acts as a denoising vector field. By learning this score, one can reverse the forward noising process to generate samples. The notes clarify the relationship between denoising and score estimation, showing that both are essentially different ways of encoding the same posterior information about the clean data.

The paper provides a detailed error decomposition for diffusion samplers, separating the errors into three components: initialization, score estimation, and numerical discretization. By using KL divergence as the primary bookkeeping metric, the author shows how to bound the sampling error by summing local one-step errors. This framework clarifies why certain discretization strategies, such as Euler–Maruyama, require specific step-size schedules to maintain accuracy, and how techniques like first-order rejection sampling can improve performance.