Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadata

Clustered Federated Learning (CFL) aims to improve model performance in heterogeneous settings by grouping clients with similar data distributions. However, existing methods face a 'CFL trilemma': they must trade off between privacy, communication efficiency, and computational cost. Metadata-based approaches are efficient but typically require sharing plaintext data representations, which violates privacy constraints. The authors introduce FLAMECHE, a framework that reformulates metadata-based clustering as a distributed Expectation-Maximization (EM) algorithm. By limiting server-side computations to additive operations, FLAMECHE allows for secure, privacy-preserving clustering without the overhead of non-linear operations on encrypted data.

FLAMECHE operates by decomposing the clustering process into an E-step (performed locally by clients) and an M-step (performed by the server). Clients use a zero-shot metadata extraction method, where they pass data through a randomly initialized neural network to obtain compact, distance-preserving representations. Because these representations are static, clients only need to upload them once. In subsequent communication rounds, the server performs additive aggregation of these metadata vectors based on client-provided cluster assignments. This approach is compatible with standard secure FL protocols like Secure Aggregation and Homomorphic Encryption, as it avoids complex, non-linear operations on the server.

This work provides a practical solution to the CFL trilemma by enabling efficient clustering that respects privacy boundaries. By using random projections for metadata extraction, the framework avoids the need for pre-training or distribution-aware design, making it highly flexible for diverse federated environments. The ability to perform clustering under encryption allows practitioners to deploy specialized models for heterogeneous client groups without exposing sensitive dataset signatures to the central server.