Research map / Optimization / Distributed and compressed
Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use accumulated quantization-error compensation instead of plain QSGD | ECQ-SGD · Gisette | loss: 1.48e{-1} -> 1.16e{-1} | Helped |
| Use accumulated quantization-error compensation instead of plain QSGD | ECQ-SGD · Gisette | communication bits: 5.88e{5} -> 5.68e{5} | Helped |
| Use accumulated quantization-error compensation instead of plain QSGD | ECQ-SGD · YearPredictionMSD | loss: 5.11e{3} -> 5.00e{3} | Helped |
| Use accumulated quantization-error compensation instead of plain QSGD | ECQ-SGD · YearPredictionMSD | communication bits: 1.17e{5} -> 1.22e{5} | Hurt |
| Set beta to 1.00 instead of 0.90 while alpha is 0.01 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 12.05% -> 8.25% | Helped |
| Increase beta from 0.90 to 0.95 while alpha is 0.01 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 12.05% -> 11.47% | Helped |
| Increase beta from 0.95 to 0.98 while alpha is 0.01 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 11.47% -> 10.69% | Helped |
| Increase beta from 0.98 to 0.99 while alpha is 0.01 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 10.69% -> 9.64% | Helped |
| Increase beta from 0.99 to 1.00 while alpha is 0.01 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 9.64% -> 8.25% | Helped |
| Reduce alpha from 0.010 to 0.001 with beta 1.00 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 8.25% -> 9.18% | Hurt |
| Reduce alpha from 0.010 to 0.003 with beta 1.00 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 8.25% -> 9.06% | Hurt |
| Increase alpha from 0.010 to 0.030 with beta 1.00 (ablation) | ECQ-SGD · CIFAR-10, ResNet-20 | error rate: 8.25% -> 8.18% | Helped |
Closest papers by meaning
- A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication (NeurIPS 2018 · similarity 0.90)
- Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification and Local Computations (NeurIPS 2019 · similarity 0.90)
- Linearly Converging Error Compensated SGD (NeurIPS 2020 · similarity 0.90)
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.90)
- DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-pass Error-Compensated Compression (ICML 2019 · similarity 0.90)
- Distributed Stochastic Optimization via Adaptive SGD (NeurIPS 2018 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.