Research map / Optimization / Distributed and compressed
Detached Error Feedback for Distributed SGD with Random Sparsification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace EF with detached error feedback | Distributed SGD with random block-wise sparsification · CIFAR-10, VGG-16, compression ratio 64 | test accuracy (%): 92.16 $\pm$ 0.06 -> 93.75 $\pm$ 0.12 | Helped |
| Use detached error feedback rather than SAEF | Distributed SGD with random block-wise sparsification · CIFAR-10, VGG-16, compression ratio 256 | test accuracy (%): 89.59 $\pm$ 0.04 -> 93.45 $\pm$ 0.11 | Helped |
| Return x_T rather than y_T (ablation) | DEF · VGG-16 on CIFAR-10 and ResNet-50 on ImageNet, early training | test accuracy: 15%; 25% | Helped |
| Return x_T rather than y_T (ablation) | DEF · Final test performance across reported CIFAR and ImageNet settings | test accuracy | Mixed |
| Set lambda to 0.1 rather than 0.3 (ablation) | DEF and DEF-A · CIFAR-10, VGG-16, compression ratio 64 | test accuracy (%): 92.83 $\pm$ 0.19; 92.78 $\pm$ 0.13 | Hurt |
| Set lambda to 0.2 rather than 0.3 (ablation) | DEF and DEF-A · CIFAR-10, VGG-16, compression ratio 64 | test accuracy (%): 93.45 $\pm$ 0.06; 93.20 $\pm$ 0.14 | No clear effect |
| Set lambda to 0.4 rather than 0.3 (ablation) | DEF and DEF-A · CIFAR-10, VGG-16, compression ratio 64 | test accuracy (%): 93.41 $\pm$ 0.12; 93.41 $\pm$ 0.20 | No clear effect |
| Set lambda to 0.6 rather than 0.3 (ablation) | DEF and DEF-A · CIFAR-10, VGG-16, compression ratio 64 | test accuracy (%): 93.26 $\pm$ 0.10; 93.11 $\pm$ 0.20 | No clear effect |
| Set lambda to 0.8 rather than 0.3 (ablation) | DEF and DEF-A · CIFAR-10, VGG-16, compression ratio 64 | test accuracy (%): 92.60 $\pm$ 0.19; 92.59 $\pm$ 0.15 | Hurt |
Closest papers by meaning
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.91)
- Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback (NeurIPS 2019 · similarity 0.91)
- Step-Ahead Error Feedback for Distributed Training with Compressed Gradient (AAAI 2021 · similarity 0.90)
- A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication (NeurIPS 2018 · similarity 0.89)
- Linearly Converging Error Compensated SGD (NeurIPS 2020 · similarity 0.89)
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.