Research map / Optimization / Nonconvex and smooth optimization
Second-Order Stochastic Optimization for Machine Learning in Linear Time
Naman Agarwal, Brian Bullins, Elad Hazan
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compare LiSSA against SVRG and SAGA in passes over the data | LiSSA · MNIST4-9, CoverType, Mushrooms, RealSIM logistic regression | log error vs passes | Improved |
| compare LiSSA against NewSamp and Newton's method in wall-clock time | LiSSA · MNIST4-9 logistic regression | log error vs time | Improved |
| compare LiSSA with NewSamp and Newton per iteration | LiSSA · MNIST4-9 logistic regression | log error vs iterations | No clear effect |
| increase S2 (Hessian inverse estimator depth) (ablation) | LiSSA · logistic regression, per-iteration convergence | convergence rate | Improved |
| compare LiSSA with accelerated first-order APCG at lambda 1e-5, 1e-6, 1e-7 | LiSSA · RealSIM (ill-conditioned) | convergence | Mixed |
Closest papers by meaning
- Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradients (JMLR 2025 · similarity 0.89)
- Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.89)
- Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits (NeurIPS 2025 · similarity 0.89)
- Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization (NeurIPS 2024 · similarity 0.87)
- Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization (NeurIPS 2025 · similarity 0.86)
- Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
