Research map / Image generation / Text-to-image diffusion
FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes why ITI-Gen prompt learning degrades T2I sample quality via cross-attention analysis (I2H/H2I switching), then proposes FairQueue (prompt queuing + attention amplification) to improve quality while keeping fairness.
Problem
Prompt-learning-based fair text-to-image generation (ITI-Gen) yields degraded sample quality; the directional alignment training objective may distort learned prompt tokens, and the mechanism behind degradation in the denoising process is unclear.
Approach
Track learned-prompt effects through cross-attention maps across denoising steps using prompt-switching analyses (I2H, H2I) and central-moment/amplitude metrics, finding that distorted learned tokens corrupt early-step global structure; then use the base prompt for early steps (prompt queuing) and scale the learned tokens' cross-attention (attention amplification) to keep fairness.
Key results
- On Smiling, FairQueue reduces FD to 0.069 from ITI-Gen's 0.124 while raising TA from 0.605 to 0.674 and lowering FID from 88.6 to 80.0.
- In a human A/B study on Smiling and High Cheekbones, annotators preferred FairQueue samples 98.7% and 97.3% of the time over ITI-Gen.
- Across CelebA single-tSA settings, FairQueue improves TA and FID over ITI-Gen for all listed tSAs, with fairness roughly matched or improved on several.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FairQueue (Prompt Queuing + Attention Amplification) | ITI-Gen · CelebA — Smiling | TA: 0.605 \pm 1.2 \text{e}^{-2} -> 0.674 \pm 1.7 \text{e}^{-2} | Helped |
| Add FairQueue (Prompt Queuing + Attention Amplification) | ITI-Gen · CelebA — Smiling | FD: 124\text{e}^{-3} \pm 9.2 \text{e}^{-3} -> 69.0 \text{e}^{-3} \pm 4.2 \text{e}^{-3} | Helped |
| Add FairQueue (Prompt Queuing + Attention Amplification) | ITI-Gen · CelebA — Young | FD: 13.1 \text{e}^{-3} \pm 8.1\text{e}^{-3} -> 15.5 \text{e}^{-3} \pm 3.8\text{e}^{-3} | Hurt |
| Add FairQueue (Prompt Queuing + Attention Amplification) | ITI-Gen · CelebA — Young | FID: 82.9 \pm 1.4 -> 75.3 \pm 2.1 | Helped |
| Add FairQueue (Prompt Queuing + Attention Amplification) | ITI-Gen · FairFace & FAIR benchmark — Gender × Age | FID: 58.24 \pm 3.4 -> 58.81\pm 3.3 | Hurt |
| Add Prompt Queuing (ablation) | FairQueue · Smiling | TA: 0.605 -> 0.674 | Helped |
| Add Attention Amplification (ablation) | FairQueue · Smiling | FD: 0.145 -> 0.069 | Helped |
| Add Prompt Queuing (ablation) | ITI-Gen with Attention Amplification · Smiling | TA: 0.610 -> 0.674 | Helped |
| Increase attention amplification factor c (ablation) | FairQueue · Smiling | fairness: c=10 | Helped |
| Increase c beyond 10 (ablation) | FairQueue · Smiling | quality: c=10 | Hurt |
| Set Prompt Queuing transition point to 0.2l (ablation) | FairQueue · Smiling | quality: 0.2l | Helped |
| Increase Prompt Queuing transition point beyond 0.2l (ablation) | FairQueue · Smiling | fairness: 0.2l | Hurt |
Surprising findings
- Attention amplification alone on ITI-Gen (without prompt queuing) yields the lowest FD (0.05) but poor sample quality, showing fairness can be gained at the cost of quality.
- Prompt queuing with the base prompt alone gives the best sample quality-fairness trade-off on some measures but poor fairness (FD 0.145 without amplification).
Limitations
- Cross-attention analysis and prompt-switching method restricted to tSAs with minimal linguistic ambiguity for the pseudo-gold standard
- Fairness still slightly worse than ITI-Gen on some tSAs (e.g., Young, Eyeglasses)
- Attention amplification factor and transition point are hyperparameters chosen via ablation
Experimental setup
Datasets: CelebA FairFace FAIR benchmark
Benchmarks: CelebA FairFace FAIR benchmark
Metrics: Fairness Discrepancy (FD) Text-Alignment (TA) FID DreamSim (DS) attention amplitude second central moment
Baselines: ITI-Gen Hard prompts (HP) Base prompt
Builds on: ITI-Gen DAAM Stable Diffusion directional loss (StyleGAN-NADA style)
Closest papers by meaning
- FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models (NeurIPS 2025 · similarity 0.89)
- Fair Text-to-Image Diffusion via Fair Mapping (AAAI 2025 · similarity 0.87)
- Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.87)
- Minority-Focused Text-to-Image Generation via Prompt Optimization (CVPR 2025 · similarity 0.87)
- Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation (CVPR 2025 · similarity 0.87)
- Devil is in the Detail: Towards Injecting Fine Details of Image Prompt in Image Generation via Conflict-free Guidance and Stratified Attention (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.