Research map / Optimization / Distributed and compressed
Compressed Decentralized Proximal Stochastic Gradient Method for Nonconvex Composite Problems with Heterogeneous Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace large-batch ProxGT-SA with small-batch (O(1)) DProxSGT with gradient tracking | FixupResNet20 sparse training · Cifar10 (500 epochs) | test accuracy | Helped |
| Replace momentum variance-reduction (DEEPSTORM) with plain stochastic proximal gradient tracking (DProxSGT) | FixupResNet20 sparse training · Cifar10 (500 epochs) | test accuracy | Helped |
| Top-k(0.3) compression of communicated parameters and gradients (CDProxSGT vs DProxSGT) | LeNet5 neural network training · FashionMNIST (100 epochs) | objective and test accuracy | No clear effect |
| Top-k(0.4) compression of communicated parameters and gradients (CDProxSGT vs DProxSGT) | FixupResNet20 neural network training · Cifar10 (500 epochs) | objective and test accuracy | No clear effect |
| Gradient tracking added to compressed decentralized SGD (CDProxSGT vs Choco-SGD) | compressed decentralized training with heterogeneous data · FashionMNIST and Cifar10 | test accuracy | Helped |
| Use large batch (256/512) with BEER instead of O(1) batch CDProxSGT | compressed training · FashionMNIST and Cifar10 | test accuracy | Hurt |
Closest papers by meaning
- Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: Joint Gradient Estimation and Tracking (ICML 2020 · similarity 0.90)
- Double Stochasticity Gazes Faster: Snap-Shot Decentralized Stochastic Gradient Tracking Methods (ICML 2024 · similarity 0.87)
- Decentralized Gradient-Free Methods for Stochastic Non-smooth Non-convex Optimization (AAAI 2024 · similarity 0.87)
- A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems (AAAI 2024 · similarity 0.87)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.87)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.