Research map / Optimization / Distributed and compressed
Revisiting Consensus Error: A Fine-grained Analysis of Local SGD under Second-order Data Heterogeneity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical analysis of Local SGD (Federated Averaging) showing its advantage over mini-batch SGD under second-order data heterogeneity, via a fine-grained study of consensus error.
Problem
Existing theory does not fully explain why Local SGD often outperforms mini-batch SGD under realistic data heterogeneity assumptions, and a second-order heterogeneity assumption has been suggested as sufficient.
Approach
A fine-grained analysis of consensus error in Local SGD under second-order data heterogeneity that confirms the second-order assumption suffices to justify local SGD's advantage over mini-batch SGD.
Key results
- Confirms that a second-order data heterogeneity assumption suffices to justify the advantage of Local SGD over mini-batch SGD (per abstract).
Experimental setup
Baselines: mini-batch SGD
Builds on: Local SGD Federated Averaging mini-batch SGD
Closest papers by meaning
- What’s in a Smoothness Constant? Tight Rates for Local SGD with Bounded Second-order Heterogeneity (NeurIPS 2026 · similarity 0.93)
- A New Theoretical Perspective on Data Heterogeneity in Federated Optimization (ICML 2024 · similarity 0.88)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.88)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.88)
- Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression (ICLR 2025 · similarity 0.88)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.