Research map / Image generation / GANs
Improving the Training of Data-Efficient GANs via Quality Aware Dynamic Discriminator Rejection Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QADDRS, a training-time discriminator rejection sampling method that dynamically rejects high-score real and low-score fake samples from each minibatch when updating the discriminator, to reduce D overfitting in data-efficient GANs.
Problem
Data-efficient GAN training suffers from discriminator overfitting, and existing remedies (augmentation, noise injection, pre-trained models) ignore the quality of samples seen during training.
Approach
Brings discriminator rejection sampling into training rather than post-hoc: sorts critic scores on real/fake minibatch samples and rejects N extreme ones per side when updating D, with N adapted each step from the ADA-style overfitting heuristic, so D does not become overconfident.
Key results
- On 256x256 low-shot datasets, StyleGAN2 + Diff-Augment FID on 100-shot Obama drops from 46.87 to 41.27 with QADDRS.
- Diffusion-Projected GAN (FastGAN) + QADDRS improves FID on AnimalFace-Dog from 17.22 to 16.57 (Table 1 entries).
- On FFHQ-100, InsGen + ADA FID improves from 45.75 to 43.67 with QADDRS; QADDRS adds negligible training time (17 to 17.25 seconds per 1K images).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add QADDRS | Diffusion-Projected GAN (FastGAN) · Obama 100-shot | FID: 10.54 -> 9.93 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN) · Grumpy Cat 100-shot | FID: 15.13 -> 14.72 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN) · Panda 100-shot | FID: 3.39 -> 3.08 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN) · Animal-Face Cat | FID: 17.86 -> 16.94 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN) · Animal-Face Dog | FID: 17.22 -> 16.57 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN Lite) · FFHQ-100 | FID: 20.65 -> 19.89 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN Lite) · FFHQ-1K | FID: 7.02 -> 6.57 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN Lite) · FFHQ-2K | FID: 5.47 -> 5.18 | Helped |
| Add QADDRS | Diffusion-Projected GAN (FastGAN Lite) · FFHQ-5K | FID: 4.74 -> 4.69 | Helped |
| Set λ to 0.25 rather than λ=0.00 (ablation) | StyleGAN2 + Diff-Augment · Obama 100-shot | FID: 46.87 -> 41.27 | Helped |
| Set λ to 0.25 rather than λ=0.50 or λ=0.75 (ablation) | StyleGAN2 + Diff-Augment · Obama 100-shot | FID: 42.40 -> 41.27; 43.32 -> 41.27 | Helped |
| Set λ to 0.25 rather than λ=0.00 (ablation) | StyleGAN2 + Diff-Augment · Animal-Face Dog | FID: 58.85 -> 51.83 | Helped |
Surprising findings
- Existing post-training DRS gives only small gains (45.22 vs 46.87 on Obama), while the training-time variant gives larger improvements.
- GAN-based methods outperform the evaluated diffusion models on low-shot datasets.
Limitations
- Gains are modest on some settings (e.g., FFHQ-100 differences are small), and the hyperparameter lambda must be set (0.25 used throughout).
- Theoretical analysis relies on idealized optimal-discriminator arguments and assumptions about supports that are not empirically verified.
Experimental setup
Datasets: Obama low-shot Grumpy Cat Panda Animal-Face Cat Animal-Face Dog FFHQ 100/1K/2K/5K
Benchmarks: Obama low-shot Grumpy Cat Panda Animal-Face Cat Animal-Face Dog FFHQ 100/1K/2K/5K
Metrics: FID FIDCLIP Precision Recall
Baselines: EDM EDM + DA Patch Diffusion TransferGAN FreezeD FakeCLR FastGAN FreGAN CHAIN NICE Projected GAN StyleGAN2 + Diff-Augment
Builds on: discriminator rejection sampling adaptive discriminator augmentation StyleGAN2 FastGAN ANDA
Closest papers by meaning
- Discriminator Rejection Sampling (ICLR 2019 · similarity 0.92)
- Improving GANs with A Dynamic Discriminator (NeurIPS 2022 · similarity 0.91)
- Collaborative Sampling in Generative Adversarial Networks (AAAI 2020 · similarity 0.89)
- On-Line Adaptative Curriculum Learning for GANs (AAAI 2019 · similarity 0.89)
- Mixed batches and symmetric discriminators for GAN training (ICML 2018 · similarity 0.89)
- Discriminative Forests Improve Generative Diversity for Generative Adversarial Networks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.