Research map / Optimization / Nonconvex and smooth optimization
On Convergence of Gradient Descent Ascent: A Tight Local Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Set the GDA stepsize ratio to r=Ω(κ), rather than a smaller ratio | GDA near a Stackelberg Equilibrium · Local convergence | convergence: r=Ω(κ); r≤κ | Helped |
| Choose r=2κ instead of the prior ratio r=Θ(κ²) | GDA near a Stackelberg Equilibrium · Local convergence | gradient complexity: Tilde{O}(κκ_x) vs Tilde{O}(κ²κ_x) | Helped |
| Increase the stepsize ratio above 2κ | GDA on quadratic functions · Quadratic NC-SC functions | convergence speed: 2κ; 200κ=2κ² | Hurt |
| Use a stepsize ratio much smaller than 2κ | GDA on quadratic functions · Quadratic NC-SC functions | convergence: 2κ | Mixed |
| Use simultaneous updates with η_x=η_y=0.001 rather than decreasing η_x and increasing the ratio | WGAN-GP · MNIST and CIFAR10 | convergence speed: η_x=η_y=0.001 | Helped |
| Decrease η_x and thereby increase the stepsize ratio | WGAN-GP · MNIST and CIFAR10 | convergent Wasserstein Distance | Hurt |
Closest papers by meaning
- On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems (ICML 2020 · similarity 0.92)
- The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimization (NeurIPS 2018 · similarity 0.91)
- GDA-AM: ON THE EFFECTIVENESS OF SOLVING MIN-IMAX OPTIMIZATION VIA ANDERSON MIXING (ICLR 2022 · similarity 0.91)
- Train simultaneously, generalize better: Stability of gradient-based minimax learners (ICML 2021 · similarity 0.90)
- Tight Analysis of Extra-gradient and Optimistic Gradient Methods For Nonconvex Minimax Problems (NeurIPS 2022 · similarity 0.89)
- Fundamental Benefit of Alternating Updates in Minimax Optimization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.