ResearchPod Summary
This paper investigates how to achieve symmetry (equivariance) in Bayesian neural networks (BNNs) without explicitly hard-coding symmetry constraints into the network architecture. While equivariant neural networks (e.g., GCNNs) are theoretically well-understood, they are rigid. Data augmentation is a flexible alternative, but its effect on the training dynamics of BNNs—and how to maximize the resulting symmetry—remains an open question.
The authors study BNNs trained with variational inference (VI) using variational distributions from the exponential family. They provide a theoretical framework showing that if the training starts from an invariant prior, the variational distribution remains invariant throughout training. They derive bounds on the equivariance error and introduce three novel techniques to enforce or boost symmetry in the variational posterior:
The study proves that under mild assumptions, data augmentation makes the evidence lower bound (ELBO) invariant, which in turn forces the variational posterior toward equivariance. The authors demonstrate that the equivariant subspace of the parameter space is preserved during gradient descent. Empirically, they show that their proposed 'orbit expansion' method consistently outperforms standard data augmentation baselines in both equivariance and overall predictive performance on image classification tasks.
This work bridges the gap between architectural symmetry constraints and data augmentation. By providing a theoretical foundation for how BNNs learn symmetries, it offers a practical, computationally efficient way to obtain uncertainty-aware, equivariant models. This is particularly valuable for scientific and medical imaging applications where data is often limited and symmetry is a known physical property.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.