ResearchPod Summary
Particle-based variational inference (ParVI) methods approximate complex target distributions by evolving a set of interacting particles. Traditional methods, such as Stein Variational Gradient Descent (SVGD), often rely on kernel-based repulsion to maintain particle diversity. However, these methods frequently suffer from variance collapse in high dimensions and mode collapse when dealing with multimodal target distributions. This paper asks whether framing particle updates as an entropic optimal transport problem can provide a more robust, globally coordinated mechanism for particle evolution.
The authors introduce Entropic Transport Descent (ETD), a family of ParVI algorithms derived from the Jordan-Kinderlehrer-Otto (JKO) proximal scheme. By lifting the optimization of particle distributions to the space of couplings and applying the Kullback-Leibler (KL) chain rule, the authors reduce each update step to an entropic optimal transport problem. This allows the algorithm to be solved efficiently using the Sinkhorn algorithm. ETD is flexible, supporting both score-based and score-free operation, and includes a parameter (tau) that controls the fidelity of the transport, allowing for a range of behaviors from semi-relaxed to fully balanced couplings.
ETD provides a global transport structure that naturally preserves multimodal distributions, outperforming existing baselines like SVGD and SGLD in high-dimensional and multimodal scenarios. The authors theoretically characterize the stationary distribution of the balanced ETD variant, demonstrating that the bias is independent of the transport cost and entropic regularization. Furthermore, they provide an importance-correction mechanism that allows the algorithm to converge exactly to the target distribution. Experimental results across Bayesian logistic regression, neural networks, and molecular energy functions confirm that ETD maintains better marginal coverage and lower variance than kernel-based alternatives.
ETD offers a robust alternative to kernel-based ParVI methods, particularly in complex inference tasks where maintaining the integrity of multiple modes is critical. By eliminating the need for score functions in its score-free variant, ETD expands the range of applicable problems to settings where gradients of the target density may be unavailable or unreliable. Its ability to handle high-dimensional spaces without the typical variance collapse seen in other methods makes it a powerful tool for modern Bayesian computation.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.