Research map / Image generation / GANs
Style Quantization for Data-Efficient GAN Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SQ-GAN, which quantizes StyleGAN2's intermediate style space W into a learnable hyperspherical codebook and uses quantization-based consistency regularization, with CLIP/optimal-transport codebook initialization, to improve limited-data GAN training.
Problem
GAN training with limited data: the sparse input latent space is poorly explored, so adjacent latents z and z+ε yield images of different realism, which harms consistency regularization of the discriminator.
Approach
Map latents to the StyleGAN-like intermediate style space, split it into sub-vectors, quantize each to a learnable codebook (with uniformity regularization on a hypersphere), and apply consistency regularization in this compact discrete proxy space; the codebook is initialized by aligning code embeddings with CLIP image features via Sinkhorn optimal transport.
Key results
- On Oxford-Dog, SQ-GAN + CBI reaches FID 35.01 and IS 12.44 versus 48.73 / 10.47 for CR with StyleGAN2.
- On FFHQ-2.5K, SQ-GAN + CBI achieves FID 22.04 versus 41.43 for CR.
- On MetFaces with ADA, SQ-GAN + ADA reaches FID 24.77 versus 29.91 for CR + ADA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add style-space quantization and knowledge-enhanced codebook initialization | StyleGAN2 · Oxford-Dog | FID: 48.73 -> 35.01 | Helped |
| Add style-space quantization and knowledge-enhanced codebook initialization | StyleGAN2 · FFHQ-2.5k | FID: 41.43 -> 22.04 | Helped |
| Add SQ-GAN to ADA | StyleGAN2 · MetFaces | FID: 28.10 -> 24.77 | Helped |
| Add SQ-GAN to ADA | StyleGAN2 · BreCaHAD | FID: 23.90 -> 22.61 | Helped |
| Add style-space quantization | StyleGAN2 · Oxford-Dog (64 × 64) | FID: 24.16 -> 18.96 | Helped |
| Use code dimension 1 instead of 4 (ablation) | SQ-GAN · Oxford-Dog | FID: 36.30 -> 37.93 | Hurt |
| Use code dimension 2 instead of 4 (ablation) | SQ-GAN · Oxford-Dog | FID: 36.30 -> 37.71 | Hurt |
| Use code dimension 8 instead of 4 (ablation) | SQ-GAN · Oxford-Dog | FID: 36.30 -> 38.60 | Hurt |
| Use code dimension 16 instead of 4 (ablation) | SQ-GAN · Oxford-Dog | FID: 36.30 -> 42.45 | Hurt |
| Use code dimension 32 instead of 4 (ablation) | SQ-GAN · Oxford-Dog | FID: 36.30 -> 45.33 | Hurt |
| Add uniformity regularization (ablation) | SQ-GAN · Oxford-Dog | FID: 40.17 -> 36.30 | Helped |
| Replace random codebook initialization with knowledge-enhanced initialization (ablation) | SQ-GAN · Oxford-Dog | FID: 36.30 -> 36.30 | No clear effect |
Surprising findings
- Even random codebook initialization remains competitive with other methods, though codebook usage drops substantially without CBI.
- On the very small MetFaces and BreCaHAD datasets, plain SQ-GAN without ADA is worse than ADA-based baselines (e.g., FID 41.12 vs 28.10 for ADA on MetFaces).
Limitations
- Gains are reported mainly on small datasets and StyleGAN2 backbone; plain SQ-GAN underperforms ADA-based methods on the smallest datasets.
- Sensitivity to code dimension: performance gains saturate beyond dimension 4 and higher dimensions cause instability.
Experimental setup
Datasets: Oxford-Dog (Oxford-IIIT Pet) FFHQ-2.5K MetFaces BreCaHAD
Benchmarks: Oxford-Dog FFHQ-2.5K MetFaces BreCaHAD
Metrics: FID IS KID
Baselines: StyleGAN2 LeCam DigGAN KD-DLGAN CR ADA DCGAN SNGAN
Builds on: StyleGAN2 consistency regularization (CR) VQ-VAE CLIP Sinkhorn divergence
Closest papers by meaning
- NICE: NoIse-modulated Consistency rEgularization for Data-Efficient GANs (NeurIPS 2023 · similarity 0.89)
- Improved Consistency Regularization for GANs (AAAI 2021 · similarity 0.89)
- Feature Quantization Improves GAN Training (ICML 2020 · similarity 0.88)
- Analyzing and Improving the Image Quality of StyleGAN (CVPR 2020 · similarity 0.88)
- MS$^3$D: A RG Flow-Based Regularization for GAN Training with Limited Data (ICML 2024 · similarity 0.87)
- CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.