Research map / Image generation / Text-to-image diffusion
FairRAG: Fair Human Generation via Fair Retrieval Augmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FairRAG, which conditions a frozen Stable Diffusion on demographically balanced reference images retrieved from an external database via a trained linear projector, improving demographic diversity in human image generation.
Problem
Text-to-image diffusion models reflect and amplify demographic biases (age, gender, skin tone) in human image generation, and prior debiasing methods do not substantially improve diversity.
Approach
Condition a frozen pre-trained latent diffusion model on retrieved real human reference images via a single linear projector mapping CLIP image embeddings into the text token space, and debias retrieval at inference with a debiased query, balanced group sampling over retrieved candidates, and a transfer instruction.
Key results
- FairRAG raises intersectional diversity from 0.188 (SDv2.1) to 0.438, versus 0.374 for the best non-RAG-free baseline Base RAG and 0.341 for TextAug.
- FairRAG reaches CLIP score 0.146 and FID 51.8, compared with 0.142 and 85.3 for SDv2.1.
- Inference is minimally slower: 3 s vs 2.8 s per image on a single A10G with 20 denoising steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add fair retrieval plus transfer instruction on top of linear-conditioned RAG (full FairRAG vs Base RAG) | Stable Diffusion v2.1 RAG · Human generation prompts | Intersectional diversity: 0.374 -> 0.438 | Helped |
| Remove debiased query from FairRAG (ablation) | FairRAG · Human generation prompts | Intersectional diversity: 0.438 -> 0.414 | Hurt |
| Remove balanced sampling from FairRAG (ablation) | FairRAG · Human generation prompts | Intersectional diversity: 0.438 -> 0.420 | Hurt |
| Remove text transfer instruction from FairRAG (ablation) | FairRAG · Human generation prompts | Intersectional diversity: 0.438 -> 0.407 | Hurt |
| Remove text transfer instruction from FairRAG (ablation) | FairRAG · Human generation prompts | Skin tone diversity: 0.416 -> 0.416 | No clear effect |
| Explicit demographic text in prompt (TextAug) vs SDv2.1 | Stable Diffusion v2.1 · Human generation prompts | Intersectional diversity: 0.188 -> 0.341 | Helped |
| Ethical phrase prompt augmentation (Interven) | Stable Diffusion v2.1 · Human generation prompts | CLIP score: 0.142 -> 0.132 | Hurt |
| Semantic guidance with uniform group prior (FairDiff) | Stable Diffusion v2.1 · Human generation prompts | Intersectional diversity: 0.188 -> 0.196 | No clear effect |
| Linear-conditioned reference retrieval (Base RAG) vs SDv2.1 | Stable Diffusion v2.1 · Human generation prompts | FID: 85.3 -> 49.4 | Helped |
Surprising findings
- A simple text-augmentation baseline (TextAug) with explicit demographic phrases is a strong baseline that prior work had not studied.
- Retrieved real images themselves are not fully diverse on skin tone and age relative to the ideal, and FairRAG remains below the retrieval-set diversity on several metrics.
Limitations
- Uses one reference image per generated image, and can produce disfigured small faces, limbs and fingers.
- Limited to human image generation and binary gender and apparent skin tone classification.
Experimental setup
Datasets: MSCOCO OpenImages-v6 FairFace (group definitions)
Benchmarks: custom profession prompt set (8 categories)
Metrics: normalized entropy demographic diversity (intersectional, ag CLIP score FID
Baselines: SDv2.1 Interven FairDiff TextAug Base RAG
Builds on: Stable Diffusion Re-Imagen Fair Diffusion Ethical Interventions CLIP
Closest papers by meaning
- FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models (NeurIPS 2025 · similarity 0.90)
- Fair Text-to-Image Diffusion via Fair Mapping (AAAI 2025 · similarity 0.90)
- Finetuning Text-to-Image Diffusion Models for Fairness (ICLR 2024 · similarity 0.87)
- AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Data (CVPR 2025 · similarity 0.86)
- On Fairness of Unified Multimodal Large Language Model for Image Generation (NeurIPS 2025 · similarity 0.86)
- FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.