Research map / Optimization / Optimizers
On Adaptive Stochastic Optimization for Streaming Data: A Newton's Method with O(dN) Operations
Antoine Godichon-Baggioni, Nicklas Werge
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Adagrad-style adaptive step instead of plain SGD | streaming SGD · Least-squares regression (d=100) | mean-squared error to optimum | Improved |
| use streaming stochastic Newton instead of Adagrad with poor initialization (r=5) | Adagrad · Least-squares regression (d=100) | mean-squared error to optimum | Improved |
| use weighted Polyak-Ruppert averaged Newton (WASSN) instead of plain stochastic Newton under poor initialization | streaming stochastic Newton · Logistic regression (d=100, r=5) | mean-squared error to optimum | Improved |
| move initialization farther from the optimum | Adagrad · Logistic regression (d=100) | mean-squared error to optimum | Hurt |
| update Hessian only partially (p=1/d, O(dN) cost) instead of fully (p=1) (ablation) | streaming stochastic Newton · Least-squares and logistic regression (d=100) | mean-squared error to optimum | No clear effect |
Closest papers by meaning
- Variance-Reduced Stochastic Gradient Descent on Streaming Data (NeurIPS 2018 · similarity 0.89)
- Do Less, Get More: Streaming Submodular Maximization with Subsampling (NeurIPS 2018 · similarity 0.87)
- Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws (ICLR 2025 · similarity 0.86)
- Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD (NeurIPS 2024 · similarity 0.86)
- Optimal and Approximate Adaptive Stochastic Quantization (NeurIPS 2024 · similarity 0.86)
- Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
