Research map / Optimization / Nonconvex and smooth optimization

Generalized Smooth Stochastic Variational Inequalities: Almost Sure Convergence and Convergence Rates

NeurIPS 2026 · Optimization › Nonconvex and smooth optimization · approach: Stochastic gradient last-iterate analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proves almost-sure and in-expectation convergence for clipped stochastic projection (SGDA-style) and clipped Korpelevich (extragradient) methods on generalized smooth (alpha-symmetric) p-quasi-sharp stochastic variational inequalities, using two samples per iteration.

Problem

Stochastic variational inequalities with non-monotone, generalized smooth operators (alpha-symmetric, not Lipschitz) as arise in adversarial training and multi-agent RL, without bounded-noise or bounded-operator assumptions.

Approach

Clip stepsizes using an independent second stochastic sample so the clipping is decoupled from the update noise; for the Korpelevich method, reuse the clipping sample from the first oracle call. This yields unbiased analysis, a.s. convergence (sum of random stepsizes diverges a.s.), and rates for alpha <= 1/2.

Key results

• Clipped stochastic projection and Korpelevich methods are shown to converge almost surely to a solution under alpha-symmetric and p-quasi-sharp assumptions, without bounded noise or bounded operator assumptions.
• For p=2 and alpha <= 1/2, last-iterate rate O(1/k) for both methods (with 1/mu^2 dependence on the sharpness parameter).
• For p>2 and alpha <= 1/2, best/average-iterate rate O(k^{-2(1-q)/p}) with 1/2<q<1; in experiments, the same-sample projection variant performs similarly to the two-sample projection, and Korpelevich offers no advantage over projection in this generalized smooth setting.

Surprising findings

• Korpelevich does not outperform projection in the generalized smooth setting, unlike the Lipschitz case.
• Same-sample clipping (biased) performs similarly to the two-sample clipped projection in experiments.

Limitations

• In-expectation rates only proved for alpha <= 1/2
• Requires p-quasi sharpness; extension to Minty or weak Minty conditions left open
• Two oracle calls per iteration

Experimental setup

Benchmarks: synthetic unconstrained minmax game

Metrics: average distance to solution set distance of averaged iterate to solution set

Baselines: clipped projection with same-sample clipping clipped Popov

Builds on: Korpelevich extragradient method stochastic gradient descent-ascent (L0,L1)-smoothness alpha-symmetric operator class p-quasi sharpness

Closest papers by meaning

• Generalized Smooth Variational Inequalities: Methods with Adaptive Stepsizes (ICML 2024 · similarity 0.94)
• Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity (ICLR 2025 · similarity 0.89)
• Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness (ICML 2025 · similarity 0.88)
• Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions (NeurIPS 2025 · similarity 0.88)
• Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity (NeurIPS 2025 · similarity 0.87)
• Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness (ICLR 2025 · similarity 0.87)