Research map / Image generation / Fast diffusion sampling
Diversity-aware Channel Pruning for StyleGAN Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a channel-pruning criterion for StyleGAN2 compression that scores channels by gradient sensitivity to latent perturbations (diversity-sensitive importance), improving sample diversity and FID before distillation.
Problem
Compressed StyleGAN generators lose sample diversity (recall) relative to the teacher, and existing channel-pruning initializations for StyleGAN compression do not account for how channels affect diversity.
Approach
Score each convolution channel by the variance of latent-perturbation-induced gradients of an L1 image-difference loss (with gradient-offset calibration), and prune low-scoring channels before distillation, so that channels responsible for sample diversity are retained.
Key results
- On FFHQ-256 the method reaches FID 6.35 vs 7.25 for StyleKD and 7.9 for CAGAN, with recall 0.339.
- On FFHQ-1024 FID is 5.80 (vs 7.19 StyleKD, 7.6 CAGAN) with recall 0.378 vs 0.277 for CAGAN.
- Reaches the previous state-of-the-art FID with about 2x fewer training iterations than StyleKD and 2.5x fewer than CAGAN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply diversity-sensitive channel pruning | StyleGAN2 compression · FFHQ-256 | FID: 7.25 -> 6.35 | Helped |
| Apply diversity-sensitive channel pruning | StyleGAN2 compression · FFHQ-1024 | Recall: 0.277 -> 0.378 | Helped |
| Apply diversity-sensitive channel pruning | StyleGAN2 compression · LSUN Church | Recall: 0.352 -> 0.372 | Helped |
| Apply diversity-sensitive channel pruning | StyleGAN2 compression · LSUN Horse | FID: 5.69 -> 5.11 | Helped |
| Apply diversity-sensitive channel pruning | StyleGAN3 compression · FFHQ | Recall: 0.376 -> 0.397 | Helped |
| Use 10 perturbation directions instead of 5 or 20 (ablation) | Diversity-aware channel pruning · FFHQ-256, 50K iterations | FIDearly: 12.46 ± 0.2; 12.47 ± 0.2 -> 12.08 ± 0.3 | Helped |
| Set perturbation strength to α=5 instead of α=1 (ablation) | Diversity-aware channel pruning · FFHQ-256, 50K iterations | FIDearly: 13.50 ± 0.3 -> 12.08 ± 0.3 | Helped |
| Use random latent directions instead of PCA directions (ablation) | Diversity-aware channel pruning · FFHQ-256 | FID: 6.35 -> 6.50 | Hurt |
| Use diversity-sensitive importance score Sσ instead of average-based score Sμ (ablation) | Diversity-aware channel pruning · FFHQ-256 | FID: 6.71 -> 6.35 | Helped |
| Use diversity-sensitive importance score Sσ instead of average-based score Sμ (ablation) | Diversity-aware channel pruning · LSUN Horse | FID: 5.36 -> 5.11 | Helped |
Surprising findings
- Random Gaussian directions for perturbation work nearly as well as GANSpace PCA directions (FID 6.50 vs 6.35).
Limitations
- Precision is slightly lower than baselines without truncation trick.
- Gains on LSUN Church and Horse are smaller and precision trade-off is evident.
Experimental setup
Datasets: FFHQ LSUN Church LSUN Horse
Benchmarks: FFHQ-256 FFHQ-1024 LSUN Church LSUN Horse FFHQ (StyleGAN3) Helen-Set55
Metrics: FID precision recall MSE LPIPS L1 distance
Baselines: CAGAN CAGAN* StyleKD StyleKD* GS Scratch
Builds on: StyleGAN2 StyleKD CAGAN GANSpace
Closest papers by meaning
- Content-Aware GAN Compression (CVPR 2021 · similarity 0.88)
- Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement (AAAI 2025 · similarity 0.87)
- Discrimination-aware Channel Pruning for Deep Neural Networks (NeurIPS 2018 · similarity 0.87)
- Diverse Rare Sample Generation with Pretrained GANs (AAAI 2025 · similarity 0.86)
- Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty (ICML 2025 · similarity 0.85)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.