Research map / Image generation / Autoregressive and latent generation
Image Generation Diversity Issues and How to Tame Them
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Image Retrieval Score (IRS), a hyperparameter-free diversity metric framed as coupon-collector image retrieval, and DiADM, a diffusion model that uses pseudo-unconditional Inception features to improve unconditional diversity without hurting FID.
Problem
Generative models often fail to cover the full training distribution, but common metrics (FID, precision, recall, coverage, density, Vendi) do not measure diversity reliably or interpretably, and existing diversity methods trade off image quality.
Approach
Frame diversity as an image retrieval problem: count how many real training images are retrieved as nearest neighbours by synthetic samples, modelled with a coupon-collector/urn formulation with Stirling numbers to get an estimate and confidence interval; normalize by the real-data retrieval score to remove feature-extractor measurement gap. DiADM replaces placeholder labels in unconditional EDM2
Key results
- Even the best evaluated ImageNet model (EDM-2-XL-512) learns only about 77% of the training data by IRS, while the smallest (ADM-256) reaches 44%.
- DiADM on ImageNet-512 reduces FID from 51.59 (EDM) to 22.28 and raises IRS from 0.09 to 0.15 relative to EDM; on FFHQ IRS rises from 0.23 to 1.51.
- Measuring gender diversity of DeepFloyd IF text-to-image outputs, IRS estimates generated diversity reaches only about 50% of a balanced reference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pseudo-label features for diversity-aware conditioning | EDM-2 on ImageNet-512 · ImageNet-512 | IRS: 0.09 -> 0.15 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on ImageNet-512 · ImageNet-512 | FID: 51.59 -> 22.28 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on FFHQ · FFHQ | IRS: 0.23 -> 1.51 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on FFHQ · FFHQ | FID: 40.92 -> 6.24 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on ChestX-ray14 · ChestX-ray14 | IRS: 0.19 -> 1.08 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on ChestX-ray14 · ChestX-ray14 | FID: 24.29 -> 6.76 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on CelebV-HQ · CelebV-HQ | IRS: 0.18 -> 0.69 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on CelebV-HQ · CelebV-HQ | FID: 68.41 -> 13.64 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on Dynamic · Dynamic | IRS: 0.60 -> 1.04 | Helped |
| use pseudo-label features for diversity-aware conditioning | EDM-2 on Dynamic · Dynamic | FID: 13.82 -> 5.56 | Helped |
| increase EDM-2 model size from XS to XL | EDM-2 on ImageNet-1k train · ImageNet-1k train | IRS: 0.46 -> 0.77 | Helped |
| increase EDM-2 model size from XS to XL | EDM-2 on ImageNet-1k train · ImageNet-1k train | FID: 3.79 -> 2.92 | Helped |
Surprising findings
- Current feature extractors (e.g., Inception, DINOv2) collapse real data in feature space, so even real datasets appear non-diverse under standard metrics (measurement gap).
- Conditional models generally achieved higher IRS than the unconditional model, suggesting conditioning should be improved.
Limitations
- Minor IRS differences are not meaningful at low sample counts and uncertainty is high.
- Diversity improvement experiments are limited to real features for generation and a restricted compute budget.
- No feature extractor was found that avoids measurement gap; the adjustment depends on real-data reference sets.
Experimental setup
Datasets: ImageNet-512 FFHQ ChestX-ray14 CelebV-HQ Dynamic
Benchmarks: ImageNet-512 FFHQ ChestX-ray14 CelebV-HQ Dynamic
Metrics: IRS FID Precision Recall Density Coverage Vendi score
Baselines: ADM DiT MAR LDM EDM-2 FID Precision and recall Coverage Density Vendi score
Builds on: EDM2 coupon collector problem Coverage and density Vendi score
Closest papers by meaning
- Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithms (ICLR 2025 · similarity 0.88)
- SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score (NeurIPS 2025 · similarity 0.88)
- The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better (NeurIPS 2024 · similarity 0.88)
- DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows (CVPR 2025 · similarity 0.87)
- Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerability (CVPR 2024 · similarity 0.87)
- FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.