OPTIMIZATION OVER COVARIANCE MATRICES WITH A PARAMETERIZED METRIC

Gradient-based optimization on the manifold of symmetric positive-definite (SPD) matrices requires choosing a Riemannian metric, which acts as an intrinsic preconditioner. Standard choices like Euclidean, Bures-Wasserstein, and affine-invariant metrics are often selected for their geometric properties rather than their algorithmic performance. This paper asks whether a more flexible, parameterized family of metrics can be tuned to better condition the Riemannian Hessian, thereby accelerating convergence for specific optimization problems.

The authors define a two-parameter family of metrics based on the Sylvester-type equation $X^pLX^q + X^qLX^p = U$. This family includes the Euclidean, Bures-Wasserstein, and affine-invariant metrics as specific members and extends beyond them. The authors analyze the condition number of the Riemannian Hessian at critical points, showing that it depends on the parameters $(p, q)$ primarily through the exponent $r = p+q$. They provide closed-form criteria to estimate the optimal $r$ and shape parameters by sampling the Hessian's diagonal in the eigenbasis of the current iterate.

The study establishes that the condition number of the Riemannian Hessian is bounded by a function of the exponent $r$. When the Hessian exhibits a power-congruence structure, the diagonal member of the family ($p=q=r/2$) attains the theoretical conditioning floor. The authors provide efficient estimation techniques to tune these parameters using only a single eigendecomposition and a few Hessian-vector products. Experiments on real covariance data and task-specific objectives demonstrate that tuning these parameters consistently outperforms standard, fixed metrics in terms of convergence speed and Hessian conditioning.

This work shifts the perspective on Riemannian metrics from purely geometric modeling to algorithmic preconditioning. By providing a principled, data-driven way to select the metric parameters, researchers can achieve faster convergence for optimization tasks involving SPD matrices, such as metric learning, Gaussian process estimation, and radar detection, without needing to manually test multiple standard geometries.