Research map / Architectures and efficiency / RNNs and normalization
The Unusual Effectiveness of Averaging in GAN Training
Yasin Yaz{\i}c{\i}, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, Vijay Chandrasekhar
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply EMA to generator parameters | Conventional GAN · CIFAR-10 | IS: 8.14 ± 0.01 -> 8.89 ± 0.12 | Improved |
| Apply EMA to generator parameters | Conventional GAN · CIFAR-10 | FID: 16.84 ± 1.00 -> 12.56 ± 0.17 | Improved |
| Apply EMA to generator parameters | Conventional GAN · STL-10 | FID: 22.33 ± 1.59 -> 19.64 ± 1.38 | Improved |
| Apply EMA to generator parameters | Conventional GAN · ImageNet | IS: 8.33 ± 0.10 -> 8.88 ± 0.14 | Improved |
| Add EMA to the base GAN | Baseline GAN · Mixture of Gaussians | Wasserstein-1 distance: 0.0419 ± 0.0054 -> 0.0251 ± 0.0026 | Improved |
| Add EMA to Optimistic Adam | Optimistic Adam · Mixture of Gaussians | Wasserstein-1 distance: 0.0494 ± 0.0041 -> 0.0431 ± 0.0042 | Improved |
| Add EMA to Consensus Optimization | Consensus Optimization · Mixture of Gaussians | Wasserstein-1 distance: 0.0286 ± 0.0034 -> 0.0252 ± 0.0035 | Improved |
| Add EMA to Zero-GP | Zero-GP · Mixture of Gaussians | Wasserstein-1 distance: 0.0363 ± 0.0018 -> 0.0246 ± 0.0068 | Improved |
| Use uniform moving average instead of no averaging | Baseline GAN · Mixture of Gaussians | Wasserstein-1 distance: 0.0419 ± 0.0054 -> 0.0274 ± 0.0016 | Improved |
| Use EMA discount β=0.9999 rather than β=0.9 (ablation) | Conventional GAN · CIFAR-10 | FID: 17.82 ± 0.43 -> 12.91 ± 0.49 | Improved |
| Set parameterized online averaging α=1 (ablation) | Conventional GAN · CIFAR-10 | IS: 8.02 ± 0.20 -> failed | Hurt |
| Set parameterized online averaging α=10 (ablation) | Conventional GAN · CIFAR-10 | FID: 17.32 ± 0.53 -> 14.55 ± 1.01 | Improved |
Closest papers by meaning
- Variational Annealing of GANs: A Langevin Perspective (ICML 2019 · similarity 0.88)
- On the Limitations of First-Order Approximation in GAN Dynamics (ICML 2018 · similarity 0.87)
- Which Training Methods for GANs do actually Converge? (ICML 2018 · similarity 0.87)
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (NeurIPS 2017 · similarity 0.87)
- A Large-Scale Study on Regularization and Normalization in GANs (ICML 2019 · similarity 0.87)
- The GAN is dead; long live the GAN! A Modern GAN Baseline (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
