Research map / Optimization / Nonconvex and smooth optimization
Stable Nonconvex-Nonconcave Training via Linear Interpolation
Thomas Pethick, Wanyun Xie, Volkan Cevher
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Lookahead-style linear interpolation on top of Adam | Adam GAN training · CIFAR10 image generation | stability (divergence) | Improved |
| use extrapolation methods (EG, RAPP) instead of the baseline despite fewer effective iterations at equal gradient computations | GAN training · CIFAR10 image generation | FID / ISC | Improved |
| use RAPP instead of EG (more inner extrapolation steps, fewer updates) | GAN training · CIFAR10 image generation | FID / ISC | Improved |
| use classical extragradient (simultaneous or alternating) under hinge loss | GAN training · CIFAR10 image generation | sample quality | Hurt |
| use EG+ instead of Adam | GAN training · CIFAR10 image generation | stability and FID: FID 16.88, ISC 8.0 | Mixed |
| set Lookahead interpolation constant to 0.5 | Lookahead · theory, comonotone problems | convergence guarantee: lambda = 0.5 | Improved |
| use many inner iterations tau in Lookahead with GDA/EG | Lookahead · theory, comonotone problems | convergence | Hurt |
Closest papers by meaning
- Stability and Generalization of Stochastic Gradient Methods for Minimax Problems (ICML 2021 · similarity 0.88)
- On the Stability Properties and the Optimization Landscape of Training Problems with Squared Loss for Neural Networks and General Nonlinear Conic Approximation Schemes (JMLR 2021 · similarity 0.88)
- Generalization and Stability of Interpolating Neural Networks with Minimal Width (JMLR 2024 · similarity 0.88)
- Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes (NeurIPS 2024 · similarity 0.87)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.87)
- Train faster, generalize better: Stability of stochastic gradient descent (ICML 2016 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
