Research map / Optimization / Distributed and compressed
Communication Efficient SGD via Gradient Sampling With Bayes Prior
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Gradient Sampling to random sampling (ablation) | Gradient compression on ResNet-50 · ImageNet | Top-1 accuracy (%): 75.65 -> 76.13 | Helped |
| Add Bayes Prior to random sampling (ablation) | Gradient compression on ResNet-50 · ImageNet | Top-1 accuracy (%): 75.65 -> 75.97 | Helped |
| Combine Gradient Sampling with Bayes Prior rather than random sampling (ablation) | Gradient compression on ResNet-50 · ImageNet | Top-1 accuracy (%): 75.65 -> 76.41 | Helped |
| Vary Bayes Prior parameter α from 0.8 to 0.95 (ablation) | Gradient Sampling with Bayes Prior on ResNet-50 · ImageNet | Top-1 accuracy (%): 76.41; 76.41; 76.39 | No clear effect |
| Use Gradient Sampling with Bayes Prior instead of SGD | ResNet-50 image classification · ImageNet | Top-1 accuracy (%): 76.44 -> 76.41 | No clear effect |
| Use Gradient Sampling with Bayes Prior instead of SGD | RetinaNet with ResNet-50 · COCO | mAP: 36.4 -> 36.1 | Hurt |
| Use Gradient Sampling with Bayes Prior instead of SGD | Faster-RCNN with ResNet-50 · PASCAL-VOC | mAP: 79.5 -> 79.0 | Hurt |
| Use Gradient Sampling with Bayes Prior instead of SGD | MobileNetV2 image classification · CIFAR-10 | Top-1 accuracy (%): 94.51 -> 94.57 | Hurt |
Closest papers by meaning
- QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding (NeurIPS 2017 · similarity 0.91)
- PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization (NeurIPS 2019 · similarity 0.91)
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.90)
- Quantized Compressive Sampling of Stochastic Gradients for Efficient Communication in Distributed Deep Learning (AAAI 2020 · similarity 0.90)
- Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback (NeurIPS 2019 · similarity 0.90)
- Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.