Research map / Optimization / Nonconvex and smooth optimization
Reducing Noise in GAN Training with Variance Reduced Extragradient
Tatjana Chavdarova, Gauthier Gidel, François Fleuret, Simon Lacoste-Julien
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace alternating stochastic gradient training with SVRE | Deep ResNet GAN · CIFAR-10 | FID: 21.70 -> 23.56 | Hurt |
| Replace stochastic extragradient with SVRE | Deep ResNet GAN · CIFAR-10 | FID: 18.65 -> 23.56 | Hurt |
| Warm-start SVRE from the best pre-divergence SE-A checkpoint | Deep ResNet GAN · CIFAR-10 | FID: 18.65 -> 16.77 | Improved |
| Replace alternating stochastic gradient training with SVRE | Deep ResNet GAN · SVHN | FID: 5.66 -> 4.81 | Improved |
| Replace stochastic extragradient with SVRE | Deep ResNet GAN · SVHN | FID: 5.14 -> 4.81 | Improved |
| Warm-start SVRE from the best pre-divergence SE-A checkpoint | Deep ResNet GAN · SVHN | FID: 5.14 -> 4.88 | Improved |
| Use SVRE instead of stochastic extragradient with Adam | GAN training · Deep architectures | training collapse rate: 100% -> did not collapse in any of the experiments | Improved |
| Combine SVRE with Adam (ablation) | SVRE · MNIST | generator second-moment estimate | Hurt |
| Add parameter averaging to SVRE (ablation) | SVRE · Stochastic bilinear game | convergence | Improved |
| Add parameter averaging to alternating stochastic gradient (ablation) | AltSGD · Stochastic bilinear game | convergence | No clear effect |
| Add SVRG-style variance reduction and extragradient updates | Stochastic bilinear game optimization · Motivating stochastic bilinear game | convergence | Improved |
| Use full-batch rather than stochastic extragradient | Extragradient · Stochastic bilinear game | convergence | Improved |
Closest papers by meaning
- TRAINING GENERATIVE ADVERSARIAL NETWORKS VIA PRIMAL-DUAL SUBGRADIENT METHODS: A LAGRANGIAN PERSPECTIVE ON GAN (ICLR 2018 · similarity 0.90)
- A Variational Inequality Perspective on Generative Adversarial Networks (ICLR 2019 · similarity 0.90)
- The Numerics of GANs (NeurIPS 2017 · similarity 0.89)
- Variational Annealing of GANs: A Langevin Perspective (ICML 2019 · similarity 0.89)
- Finding Mixed Nash Equilibria of Generative Adversarial Networks (ICML 2019 · similarity 0.88)
- MCGAN: Enhancing GAN Training with Regression-Based Generator Loss (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
