Research map / Optimization / Optimizers
Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rule
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase batch size | TTUR (Adam, AdaBelief, RMSProp) training DCGAN, WGAN-GP, BigGAN · LSUN-Bedroom, CelebA, ImageNet | number of steps to reach target FID | Helped |
| increase batch size beyond measured critical batch size | TTUR training of GANs · LSUN-Bedroom, CelebA, ImageNet | SFO complexity (N*b) | Hurt |
| estimate critical batch size from Proposition lower bound using Adam-calibrated sigma^2/epsilon^3 ratio | AdaBelief and RMSProp TTUR · DCGAN on LSUN-Bedroom | estimated vs measured critical batch size | Mixed |
| RMSProp critical batch size estimate (independent of beta1, beta2) | RMSProp TTUR · WGAN-GP on CelebA and DCGAN on LSUN-Bedroom | estimated vs measured critical batch size | Hurt |
Closest papers by meaning
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (NeurIPS 2017 · similarity 0.91)
- Small-GAN: Speeding up GAN Training using Core-Sets (ICML 2020 · similarity 0.89)
- Training Generative Adversarial Networks with Limited Data (NeurIPS 2020 · similarity 0.89)
- Progressive Growing of GANs for Improved Quality, Stability, and Variation (ICLR 2018 · similarity 0.88)
- Towards Principled Methods for Training Generative Adversarial Networks (ICLR 2017 · similarity 0.88)
- Training Generative Adversarial Networks by Solving Ordinary Differential Equations (NeurIPS 2020 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.