Smooth Reparameterizations of Functions on Simplicial Product Spaces: Applications to Probabilistic Tensor Decomposition and Functional Data Registration

Many optimization problems in machine learning and statistics, such as probabilistic tensor decomposition and functional data registration, require optimizing functions over product spaces of simplices. Traditional methods like Projected Gradient Descent (PGD) are often computationally expensive or struggle with the geometry of these constraints. This paper investigates whether these constrained problems can be reformulated as unconstrained optimization problems on smooth manifolds using specific reparameterizations, and whether this process preserves the critical points of the original objective function.

The authors propose replacing the product simplex with a smooth, elementwise strictly convex reparameterization. They generalize existing results—previously limited to the Hadamard map—to a broader class of smooth transformations that satisfy specific properties (e.g., zero at the origin, strictly positive second derivative). By constructing the Lagrangians for both the original simplex-constrained problem and the new manifold-based formulation, the authors analyze the relationship between their respective KKT conditions. They specifically distinguish between weak and strong second-order KKT points to clarify the mapping between the two spaces.

The study establishes that first-order and weak second-order KKT points on the product simplex correspond to second-order KKT points on the smooth manifold. Conversely, they show that second-order criticality on the manifold implies first-order criticality on the simplex. This theoretical framework justifies the use of Riemannian Gradient Descent (RGD) for these problems. Numerical experiments in simplex-constrained tensor factorization and shape registration demonstrate that the RGD approach provides more accurate results and smoother curve registration compared to standard PGD methods.