A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimization

Federated learning is traditionally designed for single-objective optimization, but many real-world machine learning applications require optimizing multiple conflicting objectives simultaneously (such as accuracy versus communication cost or training fairness). While multiobjective optimization (MOO) algorithms exist for centralized settings, extending them to federated learning introduces key challenges: data heterogeneity across clients, local stochastic noise, and the need to coordinate multiple objectives under communication constraints. This paper investigates how to design a federated multiobjective optimization algorithm that achieves faster convergence and greater stability than existing stochastic multi-gradient methods.

To address these challenges, the authors propose FSMGDA-M-VR, a momentum-based variance-reduced algorithm for federated multiobjective optimization. The core mechanism integrates objective-wise momentum variance-reduced estimators directly into the local client updates. Unlike previous approaches such as FSMGDA (which aggregate stochastic gradients without variance reduction) or FedCMOO (which pre-compute Pareto weights before local training), FSMGDA-M-VR allows each client to track objective-wise momentum directions across local steps. The server then aggregates these variance-reduced directions and solves a quadratic optimization problem to determine the Pareto balancing weights and construct a stable common descent direction.

Through rigorous theoretical analysis, the authors establish that the proposed algorithm successfully controls estimation error across both local steps and communication rounds. Specifically, they prove that the expected Pareto stationarity measure of a randomly selected output iterate decays at a rate of O(T^{-2/3}). This strictly improves upon the O(T^{-1/2}) convergence rate established for previous federated multiobjective methods like FSMGDA and FedCMOO. Supplementary numerical experiments on benchmark federated multiobjective tasks confirm the effectiveness and competitive performance of the algorithm.