Research map / Optimization / Nonconvex and smooth optimization
Local convergence of simultaneous min-max algorithms to differential equilibrium on Riemannian manifold
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes local linear convergence of simultaneous tau-GDA and a newly Riemannian-extended tau-SGA to differential Stackelberg/Nash equilibria on manifolds via Ostrowski theorem and spectral analysis, and tests stochastic variants on orthogonal Wasserstein GANs.
Problem
Local convergence of simultaneous min-max algorithms for zero-sum differential games where variables live on Riemannian manifolds, with rotational dynamics slowing convergence (e.g., orthogonal Wasserstein GANs).
Approach
Defines differential Stackelberg/Nash equilibria intrinsically on Riemannian manifolds, derives spectral-radius-based conditions on the learning rate ratio tau and step size gamma for tau-GDA via an Ostrowski-type theorem, and extends symplectic gradient adjustment to manifolds (tau-SGA) with an asymptotic large-tau analysis showing possibly faster rates.
Key results
- For the Example 2 toy problem, tau-GDA needs tau above about 36.18 for the local-convergence condition, while tau-SGA's bound is about 16.7, so tau-SGA can converge with smaller tau.
- On MNIST WGAN, tau-GDA at tau=5 has FID (train) 13.38 versus 7.0 at tau=10, while tau-SGA (gamma=0.02, tau=5, theta=0.075) reaches FID (train) 5.76 at T=5e5.
- tau-SGA per-iteration time is roughly 3-4 times that of tau-GDA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase τ from 5 to 10 | τ-GDA with DCGAN generator · MNIST | FID (val): 16 -> 9.1 | Helped |
| Increase τ from 10 to 20 | τ-GDA with DCGAN generator · MNIST | FID (val): 9.1 -> 9.1 | No clear effect |
| Increase training iterations from T=1×10^5 to T=5×10^5 | τ-SGA with DCGAN generator, τ=5 · MNIST | FID (val): 8.7 -> 7.6 | Helped |
| Increase training iterations from T=1×10^5 to T=5×10^5 | τ-SGA with DCGAN generator, τ=5 · MNIST | FID (train): 6.65 -> 5.76 | Helped |
| Add the SGA correction term instead of using τ-GDA at τ=5 | Stochastic min-max training with DCGAN generator · MNIST | FID (val): 16 -> 8.7 | Helped |
| Use τ=1 instead of τ=10 or 100 | Stochastic τ-GDA with linear generator · Gaussian distribution | EMD distance | Hurt |
| Use τ=100 | Stochastic τ-GDA with linear generator · Gaussian distribution | covariance error: 0.2 | Helped |
| Use τ=30 instead of τ=50 | Deterministic τ-GDA · Example 2 local convergence | convergence | Hurt |
| Add the SGA correction term | Deterministic τ-GDA on Example 2 · Example 2 local convergence | convergence rate | Helped |
Surprising findings
- tau-GDA at tau=5 has positive angle yet its objective and FID worsen well above initial values, indicating instability despite the angle signal.
Limitations
- Theory covers deterministic algorithms only; stochastic tau-GDA and tau-SGA are validated only empirically.
- Local stable region size is not explicit, so initialization guarantees are unclear.
- Asymptotic tau-SGA analysis is an approximation valid for large tau; DNE stability of tau-SGA for tau != 1 is unclear.
Experimental setup
Datasets: MNIST Fashion-MNIST synthetic Gaussian (d=5)
Benchmarks: MNIST Fashion-MNIST synthetic Gaussian EMD benchmark
Metrics: FID EMD objective value f angle covariance error
Baselines: tau-GDA extra-gradient
Builds on: Ostrowski theorem symplectic gradient adjustment gradient descent-ascent orthogonal Wasserstein GAN
Closest papers by meaning
- Local Convergence of Gradient Methods for Min-Max Games: Partial Curvature Generically Suffices (NeurIPS 2023 · similarity 0.91)
- Global Convergence to Local Minmax Equilibrium in Classes of Nonconvex Zero-Sum Games (NeurIPS 2021 · similarity 0.88)
- Adaptive Extra-Gradient Methods for Min-Max Optimization and Games (ICLR 2021 · similarity 0.87)
- Solving Neural Min-Max Games: The Role of Architecture, Initialization & Dynamics (NeurIPS 2025 · similarity 0.86)
- Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithms (NeurIPS 2024 · similarity 0.86)
- Mirror Descent-Ascent for mean-field min-max problems (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.