ResearchPod Summary
Diffusion models have taken over generative modeling, powering state-of-the-art image and video synthesis. But their origins in non-equilibrium physics and stochastic differential equations (SDEs) make them intimidating for many. This paper by Fabio De Sousa Ribeiro and Ben Glocker cuts through the fog, reframing diffusion models as Variational Diffusion Models—a natural extension of familiar latent variable models (LVMs) like VAEs. Using only graphical models and variational inference (no physics PhD required), they unify diffusion with predecessors like hierarchical LVMs (HLVMs), revealing shared math and objectives. Why care? This lowers the entry barrier, contextualizes a flood of recent advances, and equips you to innovate without reinventing the wheel.
Generative models simulate data processes to capture causal structure, not just correlations. LVMs posit hidden latents z that explain observations x via p(x|z)p(z). The Variational Autoencoder (VAE) popularized this: a neural encoder q_φ(z|x) approximates the intractable posterior p(z|x), while a decoder p_θ(x|z) reconstructs data. Training maximizes the Evidence Lower Bound (ELBO or VLB): ℒ = E[log p_θ(x|z)] - KL(q_φ(z|x) || p(z)), a tractable proxy for log p(x).
Key enablers: Amortized inference shares parameters across data for efficiency; the reparameterization trick (z = μ(x) + σ(x) ⊙ ε) enables backprop through stochastic samples.
VAEs struggle with deep hierarchies due to posterior collapse (latents become uninformative). HLVMs stack T latents z1:T in a Markov chain: p(x, z1:T) = p(x|z1) ∏ p(zt|zt-1). This boosts expressivity but amplifies inference woes.
Enter top-down inference: sample latents generatively from prior to data (zT → ... → z1 → x), contrasting bottom-up encoding. Generative feedback lets higher latents guide lower ones, stabilizing deep models. A big hurdle is the : symmetric Gaussian latents can't capture multimodal data distributions, hurting sample quality. Diffusion models fix this via structured noise.
Diffusion's signature: a forward process gradually noising data x0 to pure noise xT via Gaussian transitions q(xt|xt-1) = 𝒩(xt; √(1-βt) xt-1, βt I). Discrete-time is a Markov chain; continuous-time uses SDEs. This creates a tractable posterior q(z1:T|x0) for infinite-depth hierarchies.
The reverse process learns to denoise: p_θ(xt-1|xt) approximates the true posterior, enabling generation from noise.
Diffusion optimizes the same ELBO as LVMs: log p(x0) ≥ E_q[log p_θ(x0:T)] - KL(q(x1:T|x0) || p(x1:T)). They derive exact estimators for discrete-time (simplified denoising score matching) and continuous-time losses, proving equivalence across parameterizations (noise prediction, velocity, etc.).
Key insights: losses are invariant to noise schedules; weighting mimics importance sampling; data augmentation tightens the bound. Infinite depth? Just Ornstein-Uhlenbeck processes.
This variational lens demystifies diffusion, linking it to VAEs/HLVMs. No thermodynamics needed—everything flows from directed graphs and ELBO. It contextualizes advances (e.g., in your KB: REPA-E's latent diffusion transformers, Unified Latents' diffusion priors) and spotlights open problems like learning noise schedules.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.