Research map / Federated learning and privacy / Communication-efficient FL
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LoCoDL, a randomized primal-dual algorithm combining probabilistic local training with unbiased communication compression, achieving doubly-accelerated communication complexity for strongly convex heterogeneous distributed optimization.
Problem
Communication-efficient distributed optimization and federated learning in the heterogeneous, strongly convex setting, where clients' communication is slow and costly.
Approach
Combines Scaffnew-style probabilistic local training with compression of differences between local model estimates and a common anchor variable y, using a double lifting to a consensus problem; this works with any unbiased compressor class U(omega) instead of the permutation-based compressor required by CompressedScaffnew.
Key results
- LoCoDL with rand-k and natural compression converges faster than all compared algorithms in total communicated bits per client on logistic regression experiments.
- LoCoDL achieves uplink complexity O((sqrt(d) + d/sqrt(n)) sqrt(kappa) + d) reals, matching CompressedScaffnew while using simple independent compressors.
- ADIANA, despite better theoretical complexity for large n, is not competitive with LT-based methods in experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine probabilistic local training with compression using rand-k and Natural Compression | LoCoDL · Logistic regression on LibSVM datasets | communicated bits per client | Helped |
| Use LoCoDL instead of CompressedScaffnew | Logistic regression on LibSVM datasets · Logistic regression with n<2d and n>2d | communication efficiency | Helped |
| Use LoCoDL instead of ADIANA | Logistic regression across the reported experiments · LibSVM and MNIST datasets | communication efficiency | Helped |
| Tune rand-k parameter from k=30 to k=20 | ADIANA · Figure [ref] | convergence speed: k=20; k=30 | Helped |
| Increase learning rate γ | All algorithms · Reported experiments | convergence | Mixed |
| Vary data heterogeneity using Dirichlet parameter α | LoCoDL and ADIANA · Synthetic logistic regression, n=100 and d=10 | convergence: α = 1; α = 10 | No clear effect |
Surprising findings
- ADIANA, which has better theoretical complexity for large n, is outperformed in practice by LoCoDL and other local-training methods.
- Communication-compressed local training does not harm convergence rate until a threshold on communication probability and variance.
Limitations
- Analysis restricted to strongly convex, full-gradient (deterministic) setting; stochastic gradients and partial participation left to future work.
- Only uplink communication is analyzed; bidirectional compression not covered.
- Experiments are on small logistic regression problems rather than deep learning models.
Experimental setup
Datasets: LibSVM a5a LibSVM diabetes LibSVM w1a LibSVM australian LibSVM covtype.binary MNIST (7s vs 8s) synthetic Dirichlet-partitioned data
Benchmarks: LibSVM logistic regression (a5a, diabetes, w1a, australian, covtype.binary) MNIST 7s vs 8s logistic regression
Metrics: communicated bits per client to reach accuracy communication rounds
Baselines: DIANA ADIANA CompressedScaffnew 5GCS-CC EF21 FedCOMGATE GradSkip Scaffold
Builds on: Scaffnew DIANA CompressedScaffnew
Closest papers by meaning
- Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data (ICML 2022 · similarity 0.89)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.88)
- EFSkip: A New Error Feedback with Linear Speedup for Compressed Federated Learning with Arbitrary Data Heterogeneity (AAAI 2025 · similarity 0.87)
- Achieving Lossless Gradient Sparsification via Mapping to Alternative Space in Federated Learning (ICML 2024 · similarity 0.87)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.87)
- Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.