Research map / Optimization / Distributed and compressed
Quantized Compressive Sampling of Stochastic Gradients for Efficient Communication in Distributed Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weighted error feedback with forgetting factor to QCS (bounded residual) | QCS-SGD · convergence bound | excess term in convergence rate: O(1/sqrt(T)) -> O(1/T) | Helped |
| use QCS instead of QSG/Sparse-SGD for gradient compression | distributed SGD linear regression · synthetic linear regression | convergence time: factor of 2 at mu=0.10 vs QSGD at mu=0.05 | Helped |
| use QCS instead of QSG for compression | AlexNet distributed training · AlexNet 4 workers | time for 100 iterations: 27 seconds -> 22 seconds | Helped |
| QCS compression versus no compression | distributed training · FC/LeNet MNIST | convergence rate / final accuracy | Mixed |
Closest papers by meaning
- On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning (AAAI 2020 · similarity 0.92)
- A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication (NeurIPS 2018 · similarity 0.92)
- Indirect Stochastic Gradient Quantization and Its Application in Distributed Deep Learning (AAAI 2020 · similarity 0.91)
- Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences (NeurIPS 2024 · similarity 0.89)
- Beyond Communication Overhead: A Multilevel Monte Carlo Approach for Mitigating Compression Bias in Distributed Learning (ICML 2025 · similarity 0.88)
- Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.