Research map / Federated learning and privacy / Communication-efficient FL
Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Amplified SCAFFOLD, a federated optimization method for non-convex objectives under periodic (e.g., cyclic) client participation, with control variates and amplified updates computed over participation windows, achieving reduced communication and linear speedup.
Problem
Federated learning under non-i.i.d. periodic client participation (e.g., cyclic availability) for non-convex stochastic optimization, where prior methods lack reduced communication, linear speedup, and heterogeneity resilience.
Approach
Amplified SCAFFOLD combines amplified global updates every P-round window with control variates computed over whole participation windows, and uses a tighter analysis that takes expectation over both client participation and stochastic gradients, yielding O(eps^-2) communication for cyclic participation.
Key results
- On the synthetic objective, Amplified SCAFFOLD reached objective 0.2 in 800 communication rounds versus 1900 for SCAFFOLD and 4800 for FedAvg and Amplified FedAvg.
- Theory: for cyclic participation, communication complexity O(L K-bar eps^-2 (N/S)) versus prior O(kappa^2 eps^-4) for Amplified FedAvg.
- Ablation on Fashion-MNIST: Amplified SCAFFOLD test accuracy only drops from 84.6% to 84.45% as similarity goes from 100% to 2.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Amplified SCAFFOLD instead of SCAFFOLD | Federated learning under periodic client participation · Synthetic objective | communication rounds to objective value 0.2: 1900 -> 800 | Helped |
| Use Amplified SCAFFOLD instead of FedAvg | Federated learning under periodic client participation · Synthetic objective | communication rounds to objective value 0.2: 4800 -> 800 | Helped |
| Use Amplified SCAFFOLD instead of Amplified FedAvg | Federated learning under periodic client participation · Synthetic objective | communication rounds to objective value 0.2: 4800 -> 800 | Helped |
| Use Amplified SCAFFOLD instead of Amplified FedAvg | Federated learning under non-i.i.d. client participation · Fashion-MNIST | convergence speed | Helped |
| Use Amplified SCAFFOLD instead of Amplified FedAvg | Federated learning under non-i.i.d. client participation · CIFAR-10 | convergence speed | Helped |
| Use Amplified FedAvg instead of FedAvg | Federated learning under non-i.i.d. client participation · Synthetic objective and Fashion-MNIST | convergence speed | Helped |
| Use Amplified FedAvg instead of FedAvg | Federated learning under non-i.i.d. client participation · CIFAR-10 | convergence speed | Mixed |
| Use SCAFFOLD instead of Amplified FedAvg | Federated learning under non-i.i.d. client participation · Synthetic objective, Fashion-MNIST, and CIFAR-10 | convergence speed | Helped |
| Decrease data similarity from s=100% to s=2.5% (ablation) | Amplified SCAFFOLD · Fashion-MNIST | test accuracy: 84.6% -> 84.45% | Hurt |
| Reduce participating clients to S=5 (ablation) | Amplified SCAFFOLD · Fashion-MNIST | test accuracy | Hurt |
| Reduce participating clients to S=5 (ablation) | SCAFFOLD · Fashion-MNIST | test accuracy | Hurt |
| Increase client groups (K-bar) (ablation) | FedAvg and Amplified FedAvg · Fashion-MNIST | performance | Hurt |
Surprising findings
- Amplified FedAvg showed only slight improvement over FedAvg when trained from scratch, contrary to original evaluation which used FedAvg pretraining.
- SCAFFOLD, lacking theoretical guarantees under non-i.i.d. participation, outperformed Amplified FedAvg in all settings.
Limitations
- Requires clients to have equal participation chance within known fixed windows; does not cover clients joining or leaving dynamically.
- Worst-case communication complexity grows with the number of client groups K-bar.
- Theoretical dependence on N/S is slightly worse than SCAFFOLD's i.i.d. analysis.
Experimental setup
Datasets: Fashion-MNIST CIFAR-10 synthetic quadratic-type objective (FedAvg lower bound based)
Benchmarks: Fashion-MNIST CIFAR-10 synthetic objective
Metrics: training loss test accuracy communication rounds to target objective
Baselines: FedAvg FedProx SCAFFOLD Amplified FedAvg
Builds on: SCAFFOLD Amplified FedAvg FedAvg CyCP
Closest papers by meaning
- Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable (NeurIPS 2025 · similarity 0.92)
- Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability (NeurIPS 2024 · similarity 0.91)
- Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.90)
- Communication-Efficient Federated Learning with Accelerated Client Gradient (CVPR 2024 · similarity 0.89)
- Debiasing Federated Learning with Correlated Client Participation (ICLR 2025 · similarity 0.89)
- FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.