Research map / Optimization / Nonconvex and smooth optimization
Training GANs with Optimism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace GD with OMD | WGAN mean-learning toy game · Multivariate normal mean learning | last-iterate convergence | Helped |
| Replace GD variants with stochastic OMD | WGAN · DNA sequence generation from a PWM | KL divergence | Helped |
| Use Optimistic Adam instead of Adam | WGAN · CIFAR10 | inception score | Helped |
| Train the discriminator once per generator update instead of five times (ablation) | Stochastic OMD WGAN · DNA sequence generation from a PWM | KL divergence | Helped |
| Train the discriminator once per generator update instead of five times (ablation) | Optimistic Adam WGAN · CIFAR10 | inception score | Helped |
| Train the discriminator once per generator update instead of five times (ablation) | Adam WGAN · CIFAR10 | inception score | Hurt |
| Add gradient penalty to GD (ablation) | WGAN mean-learning toy game · Multivariate normal mean learning | limit-cycle amplitude | Mixed |
| Add momentum to GD (ablation) | WGAN mean-learning toy game · Multivariate normal mean learning | limit-cycle amplitude | Mixed |
| Add Nesterov momentum to GD (ablation) | WGAN mean-learning toy game · Multivariate normal mean learning | limit-cycle amplitude | Mixed |
| Train the discriminator multiple times between generator updates (ablation) | GD WGAN · Multivariate normal mean learning | limit-cycle amplitude | Mixed |
| Replace GD with OMD | WGAN · Covariance learning | last-iterate stability | Helped |
| Replace GD with OMD under stochastic gradients | WGAN · Stochastic covariance learning | last-iterate stability | Helped |
Closest papers by meaning
- Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile (ICLR 2019 · similarity 0.90)
- Finding Mixed Nash Equilibria of Generative Adversarial Networks (ICML 2019 · similarity 0.88)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (NeurIPS 2017 · similarity 0.87)
- A Variational Inequality Perspective on Generative Adversarial Networks (ICLR 2019 · similarity 0.87)
- Gradient descent GAN optimization is locally stable (NeurIPS 2017 · similarity 0.87)
- An Online Learning Approach to Generative Adversarial Networks (ICLR 2018 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.