Research map / Image generation / Autoregressive and latent generation
Evaluation Metrics for Conditional Image Generation
Yaniv Benny, Tomer Galanti, Sagie Benaim, Lior Wolf
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add conditional components (BCIS) instead of unconditional IS | SGAN · CIFAR10 | IS vs BCIS: IS 6.84 vs BCIS 2.71 | Improved |
| Use BCIS rather than accuracy to judge conditioning | InfoGAN · CIFAR10 | accuracy vs BCIS: Acc 65.69 vs BCIS 3.25 | Improved |
| Measure conditional metrics instead of FID alone | SGAN · MNIST | FID vs WCFID: FID 69.34 vs WCFID 609.48 | Improved |
| Apply ACGAN auxiliary classifier over CGAN | CGAN · MNIST | WCFID: 98.48 -> 91.21 | Improved |
| Apply ACGAN auxiliary classifier over CGAN | CGAN · CIFAR10 | BCFID: 6.30 -> 5.25 | Improved |
| Longer BigGAN training (100K to full) | BigGAN · ImageNet | WCFID: 11.36 -> 9.17 -> 6.33 | Improved |
| Longer BigGAN training (100K to full) | BigGAN · ImageNet | BCIS: 12.19 -> 33.71 -> 72.15 | Improved |
| Longer BigGAN training (100K to full) | BigGAN · ImageNet | accuracy: 18.87 -> 27.70 -> 51.66 | Improved |
| Label noising of conditions | IS and FID metrics · MNIST (simulation) | IS/FID: unconditional IS and FID constant while BCIS falls and WCIS rises | Mixed |
| Mode collapse simulation | BCFID vs WCFID · MNIST (simulation) | sensitivity: WCFID most noticeable, BCFID less detectable | Mixed |
| Use WCFID rather than FID for class-specific mode collapse | FID · MNIST (simulation) | sensitivity: FID as sensitive as BCFID, less than WCFID | Improved |
| Image noise, blur, swirl, PCA | BCIS vs IS · MNIST (simulation) | IS: BCIS declines more rapidly than IS | Improved |
Closest papers by meaning
- Rethinking FID: Towards a Better Evaluation Metric for Image Generation (CVPR 2024 · similarity 0.89)
- ImagenHub: Standardizing the evaluation of conditional image generation models (ICLR 2024 · similarity 0.88)
- Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerability (CVPR 2024 · similarity 0.87)
- Return of Unconditional Generation: A Self-supervised Representation Generation Method (NeurIPS 2024 · similarity 0.87)
- EvalCrafter: Benchmarking and Evaluating Large Video Generation Models (CVPR 2024 · similarity 0.87)
- GenAI Arena: An Open Evaluation Platform for Generative Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
