Research map / Federated learning and privacy / Personalized and heterogeneous FL
Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedGrEM, a robust, adaptive federated gradient EM algorithm for unsupervised mixture-model learning across tasks, with a finite-sample theory for general mixtures and applications to GMMs and mixtures of regressions.
Problem
Unsupervised federated learning where each task's data follows a mixture model, with unknown task similarity and a small fraction of adversarial or outlier tasks; existing federated EM methods lack finite-sample guarantees and robustness.
Approach
A federated gradient EM where each task performs local E-step and one gradient M-step, and the central server pools parameters via an l2-penalized (fused-type) objective toward a common center, adapting to heterogeneity and robust to contaminated tasks; analysis uses a localization trick that shrinks the uniform-convergence radius across iterations to obtain fast rates.
Key results
- On Pen-Based digits with 0% contamination, FedGrEM achieves 9.62% mis-clustering error versus 13.30% for Local-EM and 15.75% for FedEM.
- On MNIST with 24% contaminated tasks, FedGrEM keeps about 10.97% mis-clustering error while FedEM rises to 21.67%.
- In simulations, FedGrEM remains comparable to local EM/GrEM as task heterogeneity h grows, whereas FedEM and TGMM degrade below local methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedGrEM instead of Local-GrEM | GMM clustering · Pen-Based Recognition of Handwritten Digits, ε=0% | mis-clustering error rate (%): 13.20 (1.77) -> 9.62 (1.03) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · Pen-Based Recognition of Handwritten Digits, ε=6.8% | mis-clustering error rate (%): 12.96 (1.70) -> 9.31 (1.10) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · Pen-Based Recognition of Handwritten Digits, ε=13.6% | mis-clustering error rate (%): 12.62 (1.26) -> 9.04 (1.06) | Helped |
| Use FedGrEM instead of Local-EM | GMM clustering · Pen-Based Recognition of Handwritten Digits, ε=20.5% | mis-clustering error rate (%): 12.94 (1.37) -> 9.39 (1.25) | Helped |
| Use FedGrEM instead of FedEM | GMM clustering · MNIST, ε=0% | mis-clustering error rate (%): 9.97 (2.09) -> 10.63 (2.72) | Hurt |
| Use FedGrEM instead of FedEM | GMM clustering · MNIST, ε=8% | mis-clustering error rate (%): 11.97 (2.80) -> 10.64 (1.98) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · MNIST, ε=16% | mis-clustering error rate (%): 12.06 (0.93) -> 10.96 (1.96) | Helped |
| Use FedGrEM instead of Local-EM | GMM clustering · MNIST, ε=24% | mis-clustering error rate (%): 12.43 (1.11) -> 10.97 (1.59) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · Fashion-MNIST, ε=0% | mis-clustering error rate (%): 35.66 (0.64) -> 34.44 (1.98) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · Fashion-MNIST, ε=8% | mis-clustering error rate (%): 35.69 (0.65) -> 33.27 (1.96) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · Fashion-MNIST, ε=16% | mis-clustering error rate (%): 35.65 (0.70) -> 33.29 (1.86) | Helped |
| Use FedGrEM instead of Local-GrEM | GMM clustering · Fashion-MNIST, ε=24% | mis-clustering error rate (%): 35.61 (0.73) -> 33.35 (1.69) | Helped |
Surprising findings
- Pooled-EM and Pooled-GrEM perform worse than local single-task methods on real datasets, since pooling ignores task heterogeneity.
Limitations
- Theory requires good initialization, strong concavity/smoothness and uniform convergence conditions, and sample-size conditions that scale with dimension and signal strength.
- Analysis assumes contamination proportion epsilon < 1/3 and tasks in the non-outlier set share the same mixture structure.
- Experiments are limited to a single outlier configuration and small simulated settings (K=10, R=5).
Experimental setup
Datasets: Pen-Based Recognition of Handwritten Digits MNIST Fashion-MNIST
Benchmarks: Pen-Based Recognition of Handwritten Digits MNIST Fashion-MNIST
Metrics: estimation error of mixture proportions and parameters mis-clustering error rate
Baselines: Local-EM Local-GrEM FedEM TGMM FedGMM Pooled-EM Pooled-GrEM
Builds on: EM algorithm FedEM FedGMM TGMM
Closest papers by meaning
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
- TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients (AAAI 2024 · similarity 0.88)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.87)
- Federated Unlearning with Gradient Descent and Conflict Mitigation (AAAI 2025 · similarity 0.87)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.87)
- SparsyFed: Sparse Adaptive Federated Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.