Research map / Optimization / Nonconvex and smooth optimization
Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use second-order Taylor information in the extrapolation step | Extra-Newton · Second-order smooth convex minimization, deterministic oracle | convergence rate: O(1/\nRuns^2) -> O(1/\nRuns^3) | Helped |
| Use adaptive step-size based on accumulated gradient-versus-second-order prediction residuals | Extra-Newton · Second-order smooth convex minimization, stochastic and deterministic oracles | convergence rate: O(\sigma_g / \sqrt{\nRuns} + \sigma_H / \nRuns^{3/2} + 1 / \nRuns^3) | Helped |
| Decouple averaging weights from gradient weights and allow more aggressive averaging | Extra-Newton · Second-order smooth convex minimization | convergence rate: O(1 / \nRuns^3) | Helped |
| Use stochastic gradient and Hessian feedback without changing the method | Extra-Newton · Stochastic regression experiments on a1a and w1a | value convergence | Helped |
| Use Extra-Newton instead of first-order methods | Least-squares regression on a1a · a1a | value convergence | Helped |
| Use Extra-Newton instead of first-order methods | Least-squares regression on w1a · w1a | value convergence | Helped |
| Use Extra-Newton instead of OptMS | Logistic regression on a9a · a9a | linear system solutions | Helped |
| Initialize Newton farther from the solution | Newton · Logistic regression on a1a | value convergence | Hurt |
Closest papers by meaning
- Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization (NeurIPS 2024 · similarity 0.90)
- Stochastic Newton Proximal Extragradient Method (NeurIPS 2024 · similarity 0.89)
- Stochastic Zeroth-Order Optimization under Strongly Convexity and Lipschitz Hessian: Minimax Sample Complexity (NeurIPS 2024 · similarity 0.88)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · similarity 0.88)
- Last-Iterate Convergence of Single-Loop Stochastic Methods for Constrained Convex-Concave Minimax Problems (NeurIPS 2026 · similarity 0.88)
- Universal Gradient Methods for Stochastic Convex Optimization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.