Research map / Image generation / GANs
MS$^3$D: A RG Flow-Based Regularization for GAN Training with Limited Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MS$^3$D, a parameter-free regularizer that applies renormalization-group (Kadanoff block-spin) coarse-graining to the discriminator's input-gradient field and penalizes self-dissimilarity across scales, improving GAN training under limited data.
Problem
GAN training degrades when data is limited because the discriminator overfits and the gradient feedback it gives the generator becomes aggregated and unstable.
Approach
Views the discriminator's input-gradient field through renormalization-group coarse-graining and adds a multi-scale structural self-dissimilarity (MS$^3$D) loss that encourages the gradient pattern to look similar across scales, yielding a more redundant and robust gradient signal to the generator.
Key results
- On OxfordDog, StyleGAN2 + MS$^3$D + ADA reaches FID 25.94 vs 64.26 for StyleGAN2 baseline and 29.81 for CR + ADA.
- On MetFaces, + MS$^3$D reduces KID from 0.028 (StyleGAN2) to 0.017 and FID from 53.21 to 37.71.
- On FFHQ-2.5K, + MS$^3$D lowers FID from 48.11 to 33.46 (RaHinge+MS$^3$D reaches 31.95); improvements are marginal on SNGAN (45.31 to 46.14 FID).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MS^3D to StyleGAN2 | StyleGAN2 · OxfordDog | FID: 64.26 -> 47.05 | Helped |
| Add MS^3D to StyleGAN2 | StyleGAN2 · FFHQ-2.5K | FID: 48.11 -> 33.46 | Helped |
| Combine MS^3D with ADA rather than CR with ADA | StyleGAN2 · OxfordDog | FID: 29.81 -> 25.94 | Helped |
| Combine MS^3D with ADA rather than CR with ADA | StyleGAN2 · FFHQ-2.5K | FID: 19.98 -> 20.23 | Hurt |
| Add MS^3D and ADA to StyleGAN2 | StyleGAN2 · MetFaces and BreCaHAD | FID: MetFaces: 53.21 -> 23.09; BreCaHAD: 97.06 -> 20.22 | Helped |
| Add MS^3D to StyleGAN2 | StyleGAN2 · FFHQ 1K, 2K, and 4K | FID: 1K: 87.86 -> 81.30; 2K: 54.96 -> 49.17; 4K: 27.63 -> 24.17 | Helped |
| Add MS^3D to SNGAN | SNGAN · FFHQ-2.5K | FID: 45.31 -> 46.14 | Hurt |
| Apply MS^3D to both real and generated data instead of real data only (ablation) | StyleGAN2 + MS^3D · OxfordDog | FID: 47.05 -> 42.73 | Helped |
| Apply MS^3D to both real and generated data instead of real data only (ablation) | StyleGAN2 + MS^3D · FFHQ-2.5K | FID: 33.46 -> 36.31 | Hurt |
| Apply MS^3D to generated data only instead of real data only (ablation) | StyleGAN2 + MS^3D · OxfordDog and FFHQ-2.5K | FID: OxfordDog: 47.05 -> 62.40; FFHQ-2.5K: 33.46 -> 86.51 | Hurt |
| Use a Gaussian kernel instead of Kadanoff block-spin transformation (ablation) | MS^3D · OxfordDog | FID | No clear effect |
| Increase the coarse-graining factor (ablation) | MS^3D · OxfordDog | numerical stability | Hurt |
Surprising findings
- The paper reports the RaHinge loss performs well in limited data settings, contrary to prior reports.
- MS$^3$D gives little or no benefit for low-resolution SNGAN on FFHQ-2.5K, where FID slightly worsens.
Limitations
- Little benefit for low-resolution generators such as DCGAN and SNGAN (32x32).
- Regularization weight lambda must be tuned; larger values interfere with other StyleGAN2 constraints.
- Gains with transfer learning (Freeze-D) are marginal.
Experimental setup
Datasets: OxfordDog FFHQ FFHQ-2.5K MetFaces BreCaHAD Oxford-IIIT Pet
Benchmarks: OxfordDog FFHQ-2.5K FFHQ subsets (1K-70K) MetFaces BreCaHAD
Metrics: FID IS KID
Baselines: Wasserstein loss Least Squares GAN RaHinge LeCam DigGAN KD-DLGAN CR + ADA ADA Freeze-D Auxiliary rotations Dropout Noise injection
Builds on: StyleGAN2 adaptive discriminator augmentation consistency regularization DigGAN LeCam regularization
Closest papers by meaning
- A Large-Scale Study on Regularization and Normalization in GANs (ICML 2019 · similarity 0.90)
- Stabilizing Training of Generative Adversarial Networks through Regularization (NeurIPS 2017 · similarity 0.90)
- On gradient regularizers for MMD GANs (NeurIPS 2018 · similarity 0.90)
- Regularizing Generative Adversarial Networks Under Limited Data (CVPR 2021 · similarity 0.90)
- RG-GAN: Dynamic Regenerative Pruning for Data-Efficient Generative Adversarial Networks (AAAI 2024 · similarity 0.89)
- CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.