PYPM-GGD: Pitman-Yor Process Mixture with Generalized Gaussian Density using ADAM

Traditional Bayesian nonparametric (BNP) methods, such as Stochastic Variational Inference (SVI), are limited by the requirement of conjugate posteriors to maintain closed-form solutions. This paper addresses the challenge of performing large-scale learning on non-conjugate posteriors, which are often necessary for more flexible modeling but lack the analytical tractability required by standard variational inference (VI) algorithms.

The author proposes a new learning framework for Pitman-Yor Process Mixtures (PYPM) using Generalized Gaussian Density (GGD). Instead of relying on closed-form coordinate ascent, the method employs stochastic gradient ascent (SGA) to update model parameters. To ensure stable convergence in stochastic settings, the paper adapts the Adam optimization algorithm, incorporating adaptive step-sizes based on both the first and second moments of the gradient. This allows the model to learn parameters for non-conjugate distributions without needing to generate expensive Monte Carlo samples.

The proposed adaptive step-size approach (AdaSGA) demonstrates improved learning performance compared to standard constant step-size SGA and traditional SVI baselines. When applied to large-scale datasets like MIT67 and SUN397 using ResNet18 features, the method achieves clustering performance that is on-par with or superior to existing state-of-the-art deep clustering algorithms. The flexibility of the GGD allows the model to better capture non-Gaussian characteristics of the data compared to standard Gaussian mixture models.

This work provides a scalable path for applying Bayesian nonparametric models to complex, real-world datasets where conjugate priors are unavailable or inappropriate. By removing the closed-form constraint, the research enables the use of more expressive, non-conjugate likelihoods in unsupervised clustering tasks, bridging the gap between traditional Bayesian methods and modern deep learning feature extractors.