Research map / Image generation / Text-to-image diffusion

Stay Fair! Ensuring Group Fairness in Diffusion Models Across Guidance Scales

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Fairness and bias in generative models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Decomposes diffusion group bias into model and guidance-induced components and proposes StayFair, a guidance modification (debiased classifier for classifier guidance, shifted null embedding for CFG) that keeps group ratios stable across guidance scales.

Problem

Debiased diffusion models and fair generation methods are tuned for a single guidance scale, so group fairness degrades when users change the guidance scale, because guidance itself amplifies group bias.

Approach

Extends Strong Demographic Parity to guidance functions and derives a condition under which the target distribution keeps its group ratio for all guidance scales; enforces it via a Wasserstein-based fairness loss on the classifier (CG) and an attribute-shifted adaptive null prompt (CFG), so fairness is handled in the guidance step and is orthogonal to model debiasing.

Key results

• On CelebA class-conditional CG, bias range drops from 13.3% to 1.6% (smiling) and 33.6% to 6.3% (blond hair) with best fair-intra FID in both.
• On SD1.5 with CFG, bias range drops from 9.0% to 5.2%; on SD3 from 9.6% to 5.4%, with image quality roughly preserved.
• Layered on debiased FT and UCE, bias range falls from 16.7% to 6.2% (FT) and 12.8% to 5.4% (UCE).

Surprising findings

• Debiased models (FT, UCE) show a larger guidance-induced bias range than vanilla SD1.5, indicating debiasing at one scale can be inconsistent across scales.
• Standard CFG on SD3 amplifies male-associated occupations more strongly, which the authors trace to the female-biased unconditional model rather than the conditional model's male bias.

Limitations

• Only binary gender is studied; multi-attribute and intersectional bias are left to future work.
• The Gaussian log-affine theorem is idealized; for general neural guidance SDP is only an approximate sufficient condition.
• Prompt-based alpha estimator is trained on only 28 occupations and yields higher average bias than CFG.

Experimental setup

Datasets: CelebA occupation prompts (33 occupations, 5 templates)

Benchmarks: CelebA occupation-prompt gender bias on SD1.5 and SD3

Metrics: bias range average bias worst bias fair-intra FID FD_DINOv2 CLIP score aesthetic score PickScore

Baselines: CFG RW GDRO PAG FT UCE Fair Diffusion Weak Guidance

Builds on: classifier guidance classifier-free guidance strong demographic parity autoguidance Fair Diffusion UCE Wasserstein demographic parity

Closest papers by meaning

• AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Data (CVPR 2025 · similarity 0.89)
• FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models (NeurIPS 2025 · similarity 0.88)
• Balancing Act: Distribution-Guided Debiasing in Diffusion Models (CVPR 2024 · similarity 0.87)
• Finetuning Text-to-Image Diffusion Models for Fairness (ICLR 2024 · similarity 0.87)
• On Optimal Steering to Achieve Exact Fairness (NeurIPS 2025 · similarity 0.87)
• FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA (NeurIPS 2025 · similarity 0.87)