Research map / Optimization / Distributed and compressed
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rank-2 PowerSGD compression instead of uncompressed SGD | ResNet18 · CIFAR-10 | time per batch: 312 ms -> 239 ms | Helped |
| Use rank-2 PowerSGD compression instead of uncompressed SGD | ResNet18 · CIFAR-10 | test accuracy: 94.3% -> 94.4% | No clear effect |
| Use rank-4 PowerSGD compression instead of uncompressed SGD | LSTM · WikiText-2 | time per batch: 300 ms -> 134 ms | Helped |
| Use rank-4 PowerSGD compression instead of uncompressed SGD | LSTM · WikiText-2 | test perplexity: 91 -> 91 | No clear effect |
| Increase PowerSGD rank from 1 to 2 (ablation) | ResNet18 · CIFAR-10 | test accuracy: 93.6% -> 94.4% | Helped |
| Increase PowerSGD rank from 2 to 4 (ablation) | ResNet18 · CIFAR-10 | test accuracy: 94.4% -> 94.5% | Helped |
| Increase PowerSGD rank from 1 to 4 (ablation) | LSTM · WikiText-2 | test perplexity: 102 -> 91 | Helped |
| Increase PowerSGD rank from 2 to 4 (ablation) | LSTM · WikiText-2 | test perplexity: 93 -> 91 | Helped |
| Replace PowerSGD with an unbiased rank-1 compressor (ablation) | PowerSGD · CIFAR-10 | test accuracy: 93.6% -> 71.2% | Hurt |
| Replace PowerSGD with an unbiased rank-2 compressor (ablation) | PowerSGD · CIFAR-10 | test accuracy: 94.4% -> 75.9% | Hurt |
| Add warm-start initialization from the previous approximation (ablation) | PowerSGD rank 2 · CIFAR-10 | test accuracy: 94.0% -> 94.4% | Helped |
| Remove error feedback (ablation) | PowerSGD rank 4 · CIFAR-10 | test accuracy | Hurt |
Closest papers by meaning
- Error Compensated Distributed SGD Can Be Accelerated (NeurIPS 2021 · similarity 0.92)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.91)
- Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback (NeurIPS 2019 · similarity 0.91)
- Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits (NeurIPS 2026 · similarity 0.91)
- LASER: Linear Compression in Wireless Distributed Optimization (ICML 2024 · similarity 0.91)
- Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglers (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.