Research map / Optimization / Nonconvex and smooth optimization
A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| asynchronous parallelism with more workers (ASZD, linear speedup regime) | ASZD zeroth-order method · synthetic black box neural network weights | iteration (CCS) speedup: CCS 30.86 at 32 threads vs 1 thread | Helped |
| more threads in running time | ASZD zeroth-order method · synthetic black box neural network weights | running time speedup (RTS): RTS 26.12 at 32 threads vs 1 thread | Helped |
| asynchronous zeroth-order blending with 10 threads | ASZD model blending · KDD-Cup 2011 Track 1 | RMSE: 21.1241 vs 21.0004 (1st place); better than 3rd place 21.2335 but worse than 1st and 2nd | Mixed |
| larger zeroth-order step size µ allowed when gradient noise variance σ is large | ASZD · theory | allowed smoothing parameter µ | Helped |
| relaxing the linear speedup bound on staleness T from O(N^1/2) to O(N^3/4) for ASCD | asynchronous SCD · smooth unconstrained convex/nonconvex analysis | upper bound on staleness T for linear speedup | Helped |
Closest papers by meaning
- Tight Time Complexities in Parallel Stochastic Optimization with Arbitrary Computation Dynamics (ICLR 2025 · similarity 0.90)
- Faster Stochastic Optimization with Arbitrary Delays via Adaptive Asynchronous Mini-Batching (ICML 2025 · similarity 0.88)
- On the Optimal Time Complexities in Decentralized Stochastic Asynchronous Optimization (NeurIPS 2024 · similarity 0.88)
- Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations (NeurIPS 2024 · similarity 0.87)
- Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods (NeurIPS 2026 · similarity 0.87)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.