Research map / Federated learning and privacy / Communication-efficient FL
On Large-Cohort Training for Federated Learning
Zachary Charles, Zachary Garrett, Zhouyuan Huo, Sergei Shmulyian, Virginia Smith
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase cohort size from M=10 to M=800 | FedAvg on EMNIST · EMNIST | catastrophic training failure rate: 0% to 80% | Hurt |
| Apply adaptive clipping to client updates | FedAvg on EMNIST · EMNIST | catastrophic training failure rate: no catastrophic training failures occurred for any cohort size | Improved |
| Increase cohort size beyond moderate values | federated optimization methods · Multiple tasks | communication-round convergence speedup | Mixed |
| Increase cohort size from M=50 to larger cohorts | FedAdam · CIFAR-100 | test accuracy | Hurt |
| Increase cohort size | FedAdam · Stack Overflow | accuracy across test-client percentiles | Improved |
| Scale server learning rate linearly with cohort size | FedAvg · EMNIST | training stability: M > 100 | Hurt |
| Scale server learning rate by the square root of cohort size, with warmup | FedAvg · Multiple tasks | train and test accuracy | Mixed |
| Use FedLamb instead of FedAdam | federated training · Large-cohort settings | accuracy | Mixed |
| Use FedLARS instead of coordinate-wise adaptive optimization | federated training · CIFAR-100, EMNIST, and Shakespeare | accuracy | Mixed |
| Start with M=50 and double cohort size every 300 rounds up to M=800 | FedAvg and FedAdam · Multiple tasks | data efficiency | Improved |
| Normalize the server pseudo-gradient before the FedAvg server update | FedAvg · Most tasks and cohort sizes | training accuracy | Improved |
| Increase the number of local steps | FedAvg · EMNIST | test accuracy | Hurt |
Closest papers by meaning
- Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.88)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.87)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.87)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.87)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.86)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
