Research map / Image generation / Text-to-image diffusion
Reproducibility Study of "ITI-GEN: Inclusive Text-to-Image Generation"
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reproduces ITI-GEN (inclusive text-to-image tokens for Stable Diffusion), verifies most claims, finds failures with multiple/entangled attributes, and proposes training-free Hard Prompt Search with negative prompting (HPSn), plus combining it with ITI-GEN.
Problem
Text-to-image models show demographic biases; ITI-GEN claims to improve inclusiveness using reference images. This study checks whether its claims reproduce and where it fails, especially for negation, correlated attributes, and many attributes.
Approach
Reproduce ITI-GEN's claims on Stable Diffusion v1.4 and extend it with Hard Prompt Search using negative prompts (via unconditional conditioning) as a training-free alternative, and combine the two methods for complementary attribute types.
Key results
- ITI-GEN obtains FID 54.86 on CelebA (5,040 images) versus 78.34 for vanilla Stable Diffusion and 70.55 for HPS, though humans preferred HPS images in 52.24% of comparisons.
- For four binary attributes, ITI-GEN's KL divergence is about 0.3560 while HPSn's is nearly 0.
- Combining ITI-GEN fair tokens for Age/Skin tone with HPSn for Eyeglasses yields KL close to zero with FID comparable to ITI-GEN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ITI-GEN instead of vanilla Stable Diffusion | Text-to-image generation on CelebA · CelebA | FID: 78.34 -> 54.86 | Helped |
| Add negative prompting to Hard Prompt Search | Stable Diffusion · CelebA | FID: 70.55 -> 65.08 | Helped |
| Use ITI-GEN instead of HPS | Text-to-image generation · CelebA | KL divergence | Mixed |
| Add negative prompting to Hard Prompt Search | Stable Diffusion · CelebA Male × Young × Eyeglass × Smile | KL divergence: 0.032695 -> 0.000273 | Helped |
| Use ITI-GEN for multiple attributes rather than HPSn | CelebA attribute combinations · CelebA Male × Young × Eyeglass × Smile | KL divergence: 0.000273 -> 0.356023 | Hurt |
| Increase the number of attributes | ITI-GEN · CelebA attribute combinations | KL divergence: 0.051310 -> 0.378709 -> 0.356023 | Hurt |
| Apply learned tokens to different prompts without retraining | ITI-GEN · Firefighter and doctor prompts | FID: 152.94 and 157.6 | Hurt |
| Use ITI-GEN for the Skin tone attribute | Human-face generation · Skin tone | FID: 46.177 | Mixed |
| Train ITI-GEN with a gender-biased Eyeglasses reference dataset | ITI-GEN Gender × Eyeglasses · CelebA Gender × Eyeglasses | attribute accuracy | Hurt |
| Use a male-only Eyeglasses reference dataset | ITI-GEN Gender × Eyeglasses · CelebA Gender × Eyeglasses | attribute accuracy | Helped |
| Train ITI-GEN for 10 instead of 30 epochs (ablation) | ITI-GEN · CelebA Eyeglasses | KL divergence: 0.016782 -> 0.015053 | Hurt |
| Train ITI-GEN for 20 instead of 30 epochs (ablation) | ITI-GEN · CelebA Eyeglasses | KL divergence: 0.020527 -> 0.015053 | Hurt |
Surprising findings
- ITI-GEN uses gender as a proxy for attributes like Bald, Mustache and Beard, and fails to generate bald women.
- Human evaluators preferred HPS and HPSn images over ITI-GEN images, contrary to the FID ranking.
Limitations
- CLIP-based classification proved unreliable, requiring manual labeling for some attributes.
- ITI-GEN training time grows exponentially with the number of attributes and its diversity degrades with many attributes.
- HPS-based methods cannot handle attributes hard to express in text such as skin tone or colorfulness.
Experimental setup
Datasets: CelebA FAIR FairFace Landscapes HQ (LHQ) FFHQ
Benchmarks: CelebA attribute KL divergence FID vs FFHQ
Metrics: KL divergence FID human preference rate
Baselines: Hard Prompt Search (HPS) Vanilla Stable Diffusion ITI-GEN
Builds on: ITI-GEN Hard Prompt Search Stable Diffusion ControlNet CLIP
Closest papers by meaning
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.84)
- Semantic-Aware Data Augmentation for Text-to-Image Synthesis (AAAI 2024 · similarity 0.84)
- Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis (CVPR 2025 · similarity 0.83)
- Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models (AAAI 2025 · similarity 0.83)
- Discriminative Probing and Tuning for Text-to-Image Generation (CVPR 2024 · similarity 0.82)
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.