Research map / Optimization / Distributed and compressed
Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use H-SGD (G=50, I=5) instead of local SGD (P=5) | VGG-11 · CIFAR-10, 50% test accuracy | total time (s): $673.5$ -> $160.3$ | Helped |
| Use H-SGD (G=50, I=10) instead of local SGD (P=10) | VGG-11 · CIFAR-10, 50% test accuracy | total time (s): $690.2$ -> $381.1$ | Helped |
| Add local aggregation with I=5 and global period G=50 | Local SGD with P=50 · CIFAR-10, VGG-11, 50% test accuracy | total time (s): $944.3$ -> $160.3$ | Helped |
| Increase global period and decrease local period from G=50, I=5 to G=200, I=2 (ablation) | H-SGD · CIFAR-10, non-IID | test accuracy | Helped |
| Increase the number of groups (ablation) | H-SGD · Non-IID experiments | test accuracy | Hurt |
| Use group-IID grouping rather than group-non-IID grouping (ablation) | H-SGD · CIFAR-10 | test accuracy | No clear effect |
| Add local aggregation in H-SGD rather than using local SGD at the same communication time | H-SGD · CIFAR-10, FEMNIST, and CelebA | test accuracy | Helped |
Closest papers by meaning
- Multi-Level Local SGD: Distributed SGD for Heterogeneous Hierarchical Networks (ICLR 2021 · similarity 0.91)
- Revisiting Consensus Error: A Fine-grained Analysis of Local SGD under Second-order Data Heterogeneity (NeurIPS 2025 · similarity 0.90)
- Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization (NeurIPS 2019 · similarity 0.90)
- What’s in a Smoothness Constant? Tight Rates for Local SGD with Bounded Second-order Heterogeneity (NeurIPS 2026 · similarity 0.87)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.86)
- Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.