Research map / Optimization / Nonconvex and smooth optimization
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
Junchi YANG, Xiang Li, Niao He
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Nest the dual maximization in an inner loop with an adaptive stopping criterion instead of simultaneous primal-dual updates | Adaptive minimax optimization · NC-SC minimax problems | convergence with varying learning-rate ratios | Improved |
| Use simultaneous non-nested adaptive primal-dual updates without a problem-dependent time-scale ratio | Non-nested adaptive methods · Deterministic quadratic example | convergence to stationary point: r \leq L | Hurt |
| Use simultaneous GDA updates without a problem-dependent time-scale ratio | GDA · Deterministic quadratic example | convergence to stationary point: r \leq L^2 | Hurt |
| Use a learning-rate ratio of 0.01 rather than a larger ratio | Non-nested Adam, AMSGrad, and GDA · McCormick test function, stochastic | convergence: 0.01 | Hurt |
| Increase the learning-rate ratio to 0.03 | Non-nested GDA · McCormick test function, stochastic | convergence: 0.01 -> 0.03 | Improved |
| Increase the learning-rate ratio to 0.05 | Non-nested Adam and AMSGrad · McCormick test function, stochastic | convergence: 0.01 -> 0.05 | Improved |
| Use NeAda instead of Adam with fixed 15 inner-loop iterations | DRO training · Synthetic dataset | test accuracy | Improved |
| Use the same learning rates for primal and dual variables instead of different scales | NeAda · Synthetic DRO | test accuracy | No clear effect |
| Use the same learning rates for primal and dual variables instead of different scales | Adam · Synthetic DRO | test accuracy: about 68\% | Hurt |
| Use NeAda instead of Adam with the same learning rates | DRO training · MNIST | test accuracy | Improved |
Closest papers by meaning
- TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization (ICLR 2023 · similarity 0.92)
- Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization (NeurIPS 2023 · similarity 0.89)
- Global Convergence and Variance Reduction for a Class of Nonconvex-Nonconcave Minimax Problems (NeurIPS 2020 · similarity 0.89)
- Fundamental Benefit of Alternating Updates in Minimax Optimization (ICML 2024 · similarity 0.88)
- Semi-infinite Nonconvex Constrained Min-Max Optimization (NeurIPS 2025 · similarity 0.87)
- Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
