Research map / Optimization / Nonconvex and smooth optimization
A Class of Parallel Doubly Stochastic Algorithms for Large-Scale Learning
Aryan Mokhtari, Alec Koppel, Martin Takac, Alejandro Ribeiro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add second-order curvature (ARAPSA instead of RAPSA) | RAPSA · Linear regression p=500 | iterations to near optimum: ARAPSA within 10-4 by t = 300 vs RAPSA within 10 by t = 104 | Improved |
| add second-order curvature (ARAPSA instead of RAPSA) | RAPSA · MNIST binary N=1e5 | feature vectors to reach 0.1: tL = 4 × 104 -> fewer than tL = 200 | Improved |
| update fewer blocks per iteration (larger B) for RAPSA | RAPSA · Linear regression p=1024, per feature processed | convergence: B = 128 slightly better than other B | Improved |
| update fewer blocks per iteration (larger B) | RAPSA · Linear regression p=1024, per iteration | convergence: best when processors equal blocks | Hurt |
| larger B for ARAPSA hybrid step-size | ARAPSA · Linear regression p=1024 | features to reach 0.1: > 1000 -> 203 | Improved |
| larger B for ARAPSA hybrid step-size | ARAPSA · Linear regression p=1024 | iterations to reach 1e-4: 100 -> > 1000 | Hurt |
| larger B for RAPSA hybrid step-size | RAPSA · MNIST digits 0 vs 8 | iterations to reach 0.1: 74 -> 631 | Hurt |
| larger B for ARAPSA constant step-size | ARAPSA · MNIST digits 0 vs 8 | iterations to reach 0.1: 145 -> 701 | Hurt |
| asynchronous instead of synchronous RAPSA | RAPSA · Linear regression p=500 | convergence: synchronized converges faster | Hurt |
| increase clock variability in asynchronous RAPSA | Asynchronous RAPSA · MNIST logistic regression | objective F(xt): no numeric change | No clear effect |
| hybrid step-size instead of constant step-size for RAPSA | RAPSA · MNIST digits 0 vs 8 | test accuracy: 98% -> 1-2% error | Improved |
Closest papers by meaning
- Stochastic Weight Averaging in Parallel: Large-Batch Training That Generalizes Well (ICLR 2020 · similarity 0.86)
- Improved Asynchronous Parallel Optimization Analysis for Stochastic Incremental Methods (JMLR 2018 · similarity 0.85)
- Fast Asynchronous Parallel Stochastic Gradient Descent: A Lock-Free Approach with Convergence Guarantee (AAAI 2016 · similarity 0.85)
- Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees (AAAI 2021 · similarity 0.85)
- Hybrid Stochastic-Deterministic Minibatch Proximal Gradient: Less-Than-Single-Pass Optimization with Nearly Optimal Generalization (ICML 2020 · similarity 0.85)
- Fast Parallel Algorithms for Statistical Subset Selection Problems (NeurIPS 2019 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
