Research map / Optimization / Distributed and compressed
Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| inject symmetric unimodal noise into local gradients (Noisy signSGD) | signSGD with majority vote · MNIST two-layer net, heterogeneous nodes | gradient norm at convergence | Helped |
| inject noise into gradients | signSGD · MNIST full dataset, heterogeneous nodes | train and test accuracy | Helped |
| data sub-sampling for gradient evaluation | signSGD · MNIST subset, heterogeneous nodes | gradient norm at convergence | Helped |
Closest papers by meaning
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.90)
- Revisiting Consensus Error: A Fine-grained Analysis of Local SGD under Second-order Data Heterogeneity (NeurIPS 2025 · similarity 0.89)
- Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.89)
- Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates (ICLR 2024 · similarity 0.88)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.88)
- Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.