Research map / Optimization / Distributed and compressed
Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning
Tomoya Murata, Taiji Suzuki
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add parameter perturbation (BVR-L-PSGD) to BVR-L-SGD | BVR-L-SGD · heterogeneous CIFAR10 (q=0.35), 3-layer MLP | train loss | Improved |
| Add parameter perturbation (BVR-L-PSGD) to BVR-L-SGD | BVR-L-SGD · heterogeneous CIFAR10 (q=0.35), 3-layer MLP | test accuracy | Improved |
| Add perturbation to escape small-gradient-norm region | BVR-L-SGD · heterogeneous CIFAR10 (q=0.35), 3-layer MLP | escape time of stuck region | Improved |
| Use BVR-L-PSGD instead of Minibatch SGD | distributed training · heterogeneous CIFAR10 (q=0.35), 3-layer MLP | test accuracy / train loss | Improved |
| Use BVR-L-PSGD instead of Noisy Minibatch SGD | distributed training · heterogeneous CIFAR10 (q=0.35), 3-layer MLP | test accuracy / train loss | Improved |
| Add perturbation on top of BVR-L-SGD (theory) | BVR-L-SGD · nonconvex distributed optimization | communication rounds | No clear effect |
Closest papers by meaning
- Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning (ICML 2021 · similarity 0.94)
- Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization (ICML 2019 · similarity 0.89)
- Near-Optimal Distributed Minimax Optimization under the Second-Order Similarity (NeurIPS 2024 · similarity 0.88)
- Second-Order Convergence in Private Stochastic Non-Convex Optimization (NeurIPS 2025 · similarity 0.88)
- Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates (ICLR 2024 · similarity 0.88)
- An Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
