Research map / Optimization / Nonconvex and smooth optimization
Second-Order Optimization with Lazy Hessians
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reuse each Hessian for m iterations while computing fresh gradients | Cubic Newton · General non-convex optimization | arithmetic complexity: factor \sqrt{d} | Helped |
| Set the Hessian reuse phase length to d iterations (ablation) | Cubic Newton with Lazy Hessians · General non-convex optimization | arithmetic complexity: \cO\Bigl( \sqrt{d} \cdot \frac{\sqrt{L}(f(\xx_0) - f^\star)}{\varepsilon^{3/2}} \Bigr) versus \cO\Bigl( d \cdot \frac{\s | Helped |
| Update the Hessian every iteration (m = 1) (ablation) | Cubic Newton with Lazy Hessians · General non-convex optimization | arithmetic complexity: \cO\Bigl( d \cdot \frac{\sqrt{L}(f(\xx_0) - f^\star)}{\varepsilon^{3/2}} \Bigr) versus \cO\Bigl( \sqrt{d} \cdot \frac{\s | Hurt |
| Increase the Hessian reuse phase length m (ablation) | Regularized Newton with Lazy Hessians · Soft Maximum minimization | computational time | Helped |
| Use m = d as the Hessian update frequency (ablation) | Regularized Newton with Lazy Hessians · Soft Maximum minimization | computational time | Helped |
| Replace cubic regularization with quadratic gradient regularization (ablation) | Lazy Newton for convex objectives · Convex optimization | subproblem computation | Helped |
| Use a fixed regularization parameter rather than adaptive search | Cubic Newton with Lazy Hessians · General optimization | global complexity | No clear effect |
Closest papers by meaning
- Skip the Hessian, Keep the Rates: Globalized Semismooth Newton with Lazy Hessian Updates (NeurIPS 2026 · similarity 0.93)
- Second-Order Min-Max Optimization with Lazy Hessians (ICLR 2025 · similarity 0.91)
- Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization (NeurIPS 2024 · similarity 0.90)
- A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees (NeurIPS 2025 · similarity 0.90)
- Incremental Quasi-Newton Methods with Faster Superlinear Convergence Rates (AAAI 2024 · similarity 0.87)
- Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.