Research map / Optimization / Nonconvex and smooth optimization
Delay-Adaptive Step-sizes for Asynchronous Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace fixed step-size using worst-case delay upper bound with delay-adaptive step-size (Adaptive 1 or 2) | PIAG · rcv1 logistic regression | iterations to reach objective 0.3: Adaptive 1 ~1/3 and Adaptive 2 ~1/2 the iterations of fixed step-size | Helped |
| naive delay-adaptive version gamma_k = c/(tau_k+b) of fixed step-size (ablation) | PIAG and Async-BCD · toy quadratic example (theory) | divergence | Hurt |
| delay-adaptive step-size with burst delays | PIAG and Async-BCD · delay models (theory) | step-size integral sum of gamma_t: tau+1 times (Adaptive 2) larger than fixed step-size under burst delay | Helped |
| replace fixed step-size with delay-adaptive step-size | Async-BCD · rcv1 and MNIST logistic regression | convergence time | Helped |
Closest papers by meaning
- At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks? (ICLR 2020 · similarity 0.93)
- ASAP.SGD: Instance-based Adaptiveness to Staleness in Asynchronous SGD (ICML 2022 · similarity 0.91)
- Faster Stochastic Optimization with Arbitrary Delays via Adaptive Asynchronous Mini-Batching (ICML 2025 · similarity 0.89)
- Stability and Generalization of Asynchronous SGD: Sharper Bounds Beyond Lipschitz and Smoothness (NeurIPS 2024 · similarity 0.89)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.88)
- Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.