Denoising Diffusion Probabilistic Models

Deep generative models, such as GANs and VAEs, have made significant strides in image synthesis. This paper explores diffusion probabilistic models, a class of latent variable models inspired by nonequilibrium thermodynamics, to determine if they can generate high-quality images comparable to state-of-the-art methods.

The authors define a diffusion process that gradually adds Gaussian noise to data over a fixed schedule until the signal is destroyed. The model is trained to reverse this process, learning to recover the original data from noise. A key contribution is the identification of a novel connection between these diffusion models and denoising score matching with Langevin dynamics. By parameterizing the model to predict the noise added at each step rather than the mean of the posterior, the authors simplify the training objective to a weighted variational bound that significantly improves sample quality.

The proposed DDPMs achieve impressive results, including an Inception score of 9.46 and a state-of-the-art FID score of 3.17 on the unconditional CIFAR10 dataset. The authors show that their models naturally support a progressive lossy decompression scheme, where large-scale image features are generated first, followed by finer details. This process is interpreted as a generalization of autoregressive decoding, providing a new perspective on how these models learn to structure data.

This work establishes diffusion models as a powerful and viable alternative to GANs and other generative architectures. By demonstrating that simple Gaussian diffusion can produce high-fidelity images, the paper opens new avenues for research into generative modeling, data compression, and representation learning. The connection to denoising score matching provides a theoretical foundation that helps explain the effectiveness of these models and suggests future improvements.