Research map / Optimization / Distributed and compressed
Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ordinary sparsification with matrix-smoothness-aware sparsification | DCGD · Strongly convex distributed optimization; interpolation | iteration complexity: up to \min(n,d) times speedup | Helped |
| Replace ordinary sparsification with matrix-smoothness-aware sparsification | DIANA · Strongly convex distributed optimization | iteration complexity: up to \min(n,d) times speedup | Helped |
| Replace ordinary sparsification with matrix-smoothness-aware sparsification | ADIANA · Strongly convex distributed optimization | iteration complexity: factor of ${\cal O}(\sqrt{d})$ in high compression; factor of ${\cal O}(\sqrt{\min(n,d)})$ in low compression | Helped |
| Use matrix-aware rather than standard uniform sparsification | DIANA+ · LibSVM logistic regression; six datasets | convergence speed | Helped |
| Use importance sampling rather than uniform sampling (ablation) | DIANA+ · LibSVM logistic regression; six datasets | convergence speed | Helped |
| Use matrix-aware sparsification | DCGD · LibSVM logistic regression; six datasets | convergence speed | Helped |
| Use matrix-aware sparsification | DIANA · LibSVM logistic regression; six datasets | convergence speed | Helped |
| Use matrix-aware sparsification | ADIANA · LibSVM logistic regression; six datasets | convergence speed | Helped |
| Add acceleration (ablation) | DIANA+ · LibSVM logistic regression; six datasets | convergence speed | Helped |
| Add variance reduction (ablation) | DCGD+ · LibSVM logistic regression; six datasets | convergence stability | Helped |
| Reduce sparsification level below a threshold (ablation) | DIANA+ · LibSVM logistic regression; six datasets | iteration complexity | Hurt |
| Use importance sampling instead of uniform sampling at low sparsification levels (ablation) | DIANA+ · LibSVM logistic regression; six datasets | iteration complexity | Helped |
Closest papers by meaning
- Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniques (NeurIPS 2022 · similarity 0.92)
- Distributed Optimization for Overparameterized Problems: Achieving Optimal Dimension Independent Communication Complexity (NeurIPS 2022 · similarity 0.90)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.88)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.88)
- Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates (ICLR 2024 · similarity 0.87)
- Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.