Research map / Image generation / GANs
CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CHAIN, a modified batch normalization for GAN discriminators (zero-mean regularization plus Lipschitz-constrained adaptive RMS normalization) to curb discriminator overfitting in data-efficient GANs.
Problem
GANs trained with limited data suffer from discriminator overfitting and unstable training, and batch normalization, despite improving generalization, is rarely used in discriminators because it causes gradient explosion.
Approach
Replace BN's centering step with a soft zero-mean regularization loss, constrain the scaling step to be 1-Lipschitz by dividing by the root mean square and multiplying by the minimum RMS, and adaptively interpolate normalized and unnormalized features based on a real-data overfitting signal.
Key results
- On 10% CIFAR-10 with OmniGAN, CHAIN improves tFID from 22.24 (baseline) to 8.27.
- On 10% CIFAR-100 with BigGAN, CHAIN reduces tFID from 50.79 to 13.13 and vFID from 55.04 to 18.00.
- On five 100-shot StyleGAN2 datasets, ADA+CHAIN achieves e.g. 20.94 FID on Obama versus 45.69 for ADA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CHAIN to BigGAN | BigGAN · CIFAR-10 10% data | tFID: 12.64 -> 12.02 | Helped |
| Add CHAIN to BigGAN | BigGAN · CIFAR-100 10% data | tFID: 22.40 -> 13.13 | Helped |
| Add ADA and CHAIN to BigGAN | BigGAN · ImageNet 2.5%, 5%, and 10% data | tFID: 28.79 -> 23.01; 22.35 -> 16.14; 20.32 -> 12.91 | Helped |
| Add CHAIN to FastGAN-Dbig | FastGAN-Dbig · Seven high-resolution few-shot datasets | FID: 171.35˘6.91 -> 78.62˘1.21; 165.64˘11.47 -> 82.47˘2.82; 76.02˘5.37 -> 46.27˘0.36; 68.63˘1.18 -> 58.98˘1.59; 37.38˘1.73 -> | Mixed |
| Add CHAIN to StyleGAN2 | StyleGAN2 · Five low-shot datasets | FID: 80.20 -> 28.72; 48.90 -> 27.21; 34.27 -> 9.51; 71.71 -> 38.93; 131.90 -> 53.27 | Mixed |
| Add adaptive augmentation with CHAIN | StyleGAN2 · AnimalFace Cat | FID: 38.93 -> 19.74 | Helped |
| Add data augmentation with CHAIN | StyleGAN2 · AnimalFace Dog | FID: 53.27 -> 30.88 | Helped |
| Add 0-mean regularization (ablation) | CHAIN with OmniGAN · CIFAR-10 10% data | tFID: 9.20 -> 8.27 | Helped |
| Add adaptive root mean square normalization (ablation) | CHAIN with OmniGAN · CIFAR-10 10% data | tFID: 12.87 -> 8.27 | Helped |
| Remove the centering step (ablation) | CHAIN with OmniGAN · CIFAR-10 10% data | tFID: 8.99 -> 8.27 | Helped |
| Add the Lipschitzness constraint (ablation) | CHAIN with OmniGAN · CIFAR-10 10% data | tFID: 22.14 -> 8.27 | Helped |
| Use running cumulative statistics instead of batch statistics (ablation) | CHAIN with OmniGAN · CIFAR-10 10% data | tFID: 8.51 -> 8.27 | Helped |
Surprising findings
- Plain batch normalization in the discriminator worsens FID substantially (e.g., BigGAN 10% CIFAR-100 tFID 50.79 to 55.83), consistent with prior observations that BN hurts GAN discriminators.
- Replacing centering with adaptive interpolation (0C/A0C) alone degrades results versus baseline on CIFAR-10 with OmniGAN.
Limitations
- Batch statistics depend on minibatch size, motivating running cumulative statistics for multi-GPU high-resolution training.
- Requires a hyperparameter threshold tau and adaptive step size for the overfitting signal; the theory is developed for an idealized setting with a Lipschitz-constant argument rather than full end-to-end analysis.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet Obama Panda Grumpy Cat AnimalFace Shells Skulls AnimeFace Pokemon ArtPainting
Benchmarks: CIFAR-10 CIFAR-100 ImageNet 5 low-shot datasets 7 high-resolution few-shot datasets
Metrics: IS tFID vFID FID
Baselines: BigGAN OmniGAN DA ADA LeCam KDDLGAN DigGAN FastGAN FreGAN StyleGAN2 GenCo InsGen
Builds on: BigGAN OmniGAN FastGAN StyleGAN2 DiffAugment (DA) ADA
Closest papers by meaning
- A Large-Scale Study on Regularization and Normalization in GANs (ICML 2019 · similarity 0.89)
- Which Training Methods for GANs do actually Converge? (ICML 2018 · similarity 0.89)
- Improving Generalization and Stability of Generative Adversarial Networks (ICLR 2019 · similarity 0.89)
- Sparsity Aware Normalization for GANs (AAAI 2021 · similarity 0.89)
- MS$^3$D: A RG Flow-Based Regularization for GAN Training with Limited Data (ICML 2024 · similarity 0.88)
- RG-GAN: Dynamic Regenerative Pruning for Data-Efficient Generative Adversarial Networks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.