Research map / Federated learning and privacy / Communication-efficient FL
Stochastic Controlled Averaging for Federated Learning with Communication Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits SCAFFOLD with a single-variable uplink formulation and proposes SCALLION (unbiased compression) and SCAFCOM (biased compression with local momentum) for compressed federated learning under arbitrary heterogeneity and partial participation.
Problem
Communication compression in federated learning interacts badly with data heterogeneity, partial participation and local updates; existing compressed FL methods either fail under arbitrary heterogeneity/partial participation or need stringent compressor or gradient assumptions.
Approach
Reformulate SCAFFOLD so each client uplinks only one increment variable (halving uplink cost), then compress that increment: unbiased compression with a scaling factor alpha (SCALLION), or biased contractive compression with a local momentum variable (SCAFCOM). Compressing increments makes compression error vanish near stationarity, so no extra assumptions on heterogeneity or compression error are
Key results
- SCAFCOM with Top-0.01 compression reaches test accuracy very close to full-precision SCAFFOLD on MNIST and FMNIST, and Top-0.05 essentially matches it, saving roughly 20-100x uplink communication.
- SCALLION with 2-bit and 4-bit random dithering basically overlaps SCAFFOLD curves and outperforms FedCOMGATE under the same bit budget.
- Theory: SCALLION and SCAFCOM achieve asymptotic communication and computation complexities that match or improve prior compressed FL methods without bounded gradient or compression-error assumptions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCALLION instead of FedCOMGATE with unbiased compression | Non-convex FL · Non-convex FL, unbiased compression, all clients participate | asymptotic communication complexity | No clear effect |
| Use SCALLION instead of FedCOMGATE with unbiased compression | Non-convex FL · Non-convex FL, unbiased compression, all clients participate | asymptotic computation complexity | No clear effect |
| Use SCAFCOM instead of Fed-EF with biased compression | Non-convex FL · Non-convex FL, biased compression, all clients participate | asymptotic communication complexity: \frac{1}{(1-q)^2\epsilon} -> \frac{1}{(1-q)\epsilon} | Helped |
| Use SCAFCOM instead of SCALLION with unbiased compression | Non-convex FL · Non-convex FL, unbiased compression, all clients participate | asymptotic computation complexity: \frac{1+\omega}{N\epsilon^2} -> \frac{1}{N\epsilon^2} | Helped |
| Add local momentum to the compressed controlled-averaging method | SCAFCOM · MNIST and Fashion MNIST | test accuracy | Helped |
| Use β=0.2 rather than other tested momentum values (ablation) | SCAFCOM with Top-0.01 · MNIST and Fashion MNIST | test accuracy | Helped |
| Use α>0.5 for 2-bit random dithering (ablation) | SCALLION · MNIST and Fashion MNIST | training stability | Hurt |
| Increase random-dithering precision from 2 bits to 4 bits (ablation) | SCALLION · MNIST and Fashion MNIST | test accuracy | Helped |
| Increase Top-r sparsification from Top-0.01 to Top-0.05 (ablation) | SCAFCOM · MNIST and Fashion MNIST | test accuracy relative to full-precision SCAFFOLD | Helped |
Surprising findings
- For SCALLION with low-bit random dithering, scaling factor alpha above 0.5 makes training unstable, so a small alpha (0.1) is needed.
Limitations
- Experiments are limited to small fully-connected networks on MNIST and Fashion-MNIST with simulated compression.
- Unbiased-compression theory needs a small scaling factor alpha for stability with low-bit quantization.
- SCAFCOM applied to unbiased compressors uses a looser rate than the unbiased analysis (refinement omitted).
Experimental setup
Datasets: MNIST Fashion-MNIST
Benchmarks: MNIST Fashion-MNIST
Metrics: test accuracy training loss
Baselines: FedCOMGATE Fed-EF SCAFFOLD FedAvg (Fed-SGD)
Builds on: SCAFFOLD FedAvg EF21 FedCOMGATE
Closest papers by meaning
- Robust Federated Learning Against Adaptive Compression (ICML 2026 · similarity 0.91)
- EFSkip: A New Error Feedback with Linear Speedup for Compressed Federated Learning with Arbitrary Data Heterogeneity (AAAI 2025 · similarity 0.91)
- z-SignFedAvg: A Unified Stochastic Sign-Based Compression for Federated Learning (AAAI 2024 · similarity 0.88)
- FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model Updates (ICML 2025 · similarity 0.87)
- Achieving Lossless Gradient Sparsification via Mapping to Alternative Space in Federated Learning (ICML 2024 · similarity 0.87)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.