Research map / Optimization / Nonconvex and smooth optimization

Skip the Hessian, Keep the Rates: Globalized Semismooth Newton with Lazy Hessian Updates

NeurIPS 2026 · Optimization › Nonconvex and smooth optimization · approach: Adaptive first-order convex methods · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GLAd-SSN, a globalized semismooth Newton method with lazy Hessian updates (Hessian refreshed every m iterations) and gradient regularization, giving global rates plus local superlinear and gamma-order convergence in Hilbert spaces.

Problem

Second-order methods for composite minimization with nonsmooth (semismooth) derivatives lack global convergence rates and local superlinear guarantees, and their per-iteration Hessian cost is high.

Approach

A proximal regularized Newton step that reuses a stale generalized Hessian for m iterations, with gradient-norm-dependent regularization lambda = 4^j Lambda_k ||F'(x_k)||^p chosen by a backtracking line search, yielding global O(1/sqrt k) rates and superlinear / gamma-order local rates under semismoothness.

Key results

• Global nonasymptotic rate: min subgradient norm on trajectory is at most 4 sqrt(lambda_bar F_0 / k), with at most ceil(k/m) Hessian evaluations up to iteration k.
• Under PL condition, objective values converge linearly with factor exp(-mu k/(mu+8 lambda_bar)), and superlinearly once lambda_k goes to 0.
• Under gamma-order semismoothness with m=1, the gradient norm converges with (1+2gamma)/(1+gamma)-order superlinear rate; experiments show GLAd-SSN is an order of magnitude better in iteration count than Adam on Lipschitz-constrained networks, and laziness saves time.

Limitations

• Superlinear convergence requires the sequence to converge to a solution and local strong convexity or PL assumptions.
• Code is promised only at acceptance; no release at submission.
• Authors note further efficiency aspects (sketching, quasi-Newton) are left to future work.

Experimental setup

Metrics: iteration count running time Lipschitz constant estimate non-negativity constraint violation objective value

Baselines: Adam LeAP-SSN gradient descent with Armijo linesearch GLAd-SSN without laziness

Builds on: LeAP-SSN universal gradient-regularized Newton method lazy Hessian updates (Doikov et al.) Dennis-Moré condition

Closest papers by meaning

• Second-Order Optimization with Lazy Hessians (ICML 2023 · similarity 0.93)
• A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees (NeurIPS 2025 · similarity 0.90)
• Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization (NeurIPS 2024 · similarity 0.89)
• Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.88)
• Stochastic Newton Proximal Extragradient Method (NeurIPS 2024 · similarity 0.88)
• Incremental Quasi-Newton Methods with Faster Superlinear Convergence Rates (AAAI 2024 · similarity 0.88)