Research map / Optimization / Distributed and compressed
Step-Ahead Error Feedback for Distributed Training with Compressed Gradient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAEF without error averaging (p=∞) instead of local error feedback | ResNet-56 · CIFAR-100 | Top-1 testing accuracy (%): 50.00 $\pm$ 0.70 -> 60.59 $\pm$ 0.28 | Helped |
| Use SAEF without error averaging (p=∞) instead of local error feedback | ResNet-56 · CIFAR-100 | Top-5 testing accuracy (%): 50.06 $\pm$ 1.17 -> 60.21 $\pm$ 0.64 | Helped |
| Use SAEF without error averaging (p=∞) instead of local error feedback | ResNet-56 · CIFAR-100 | Top-10 testing accuracy (%): 51.11 $\pm$ 0.24 -> 57.78 $\pm$ 0.21 | Helped |
| Add error averaging every 40 iterations to SAEF (ablation) | SAEF · CIFAR-100 | Top-1 testing accuracy (%): 60.59 $\pm$ 0.28 -> 62.83 $\pm$ 0.27 | Helped |
| Add error averaging every 20 iterations to SAEF (ablation) | SAEF · CIFAR-100 | Top-1 testing accuracy (%): 60.59 $\pm$ 0.28 -> 63.96 $\pm$ 0.35 | Helped |
| Add error averaging every 10 iterations to SAEF (ablation) | SAEF · CIFAR-100 | Top-1 testing accuracy (%): 60.59 $\pm$ 0.28 -> 64.55 $\pm$ 0.20 | Helped |
| Add error averaging every 5 iterations to SAEF (ablation) | SAEF · CIFAR-100 | Top-1 testing accuracy (%): 60.59 $\pm$ 0.28 -> 65.62 $\pm$ 0.24 | Helped |
| Add error averaging every iteration to SAEF (ablation) | SAEF · CIFAR-100 | Top-1 testing accuracy (%): 60.59 $\pm$ 0.28 -> 65.89 $\pm$ 0.32 | Helped |
| Use SAEF instead of local error feedback with compressed gradients | Distributed ResNet training · CIFAR-100 and ImageNet | Training loss | Helped |
| Add error averaging to SAEF (ablation) | Distributed training · CIFAR-100 and ImageNet | Training convergence | Helped |
| Add error averaging to local error feedback (ablation) | Local error feedback · CIFAR-100 and ImageNet | Training convergence | Hurt |
Closest papers by meaning
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.91)
- Detached Error Feedback for Distributed SGD with Random Sparsification (ICML 2022 · similarity 0.90)
- On the Discrepancy between the Theoretical Analysis and Practical Implementations of Compressed Communication for Distributed Deep Learning (AAAI 2020 · similarity 0.90)
- ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training (NeurIPS 2020 · similarity 0.89)
- Sketched Adaptive Distributed Deep Learning: A Sharp Convergence Analysis (NeurIPS 2025 · similarity 0.88)
- Tight analyses of first-order methods with error feedback (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.