Research map / Image generation / Text-to-image diffusion
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OpenBias, a modular LLM + VQA pipeline that discovers, assesses and ranks open-set biases in text-to-image models without a predefined bias list, applied to Stable Diffusion 1.5/2/XL.
Problem
Existing bias detection for text-to-image generative models relies on closed sets of predefined biases (e.g., gender, race), so unanticipated biases such as object attributes or novel person attributes go undetected.
Approach
Use an LLM to propose candidate biases, classes and VQA questions from real captions, generate images with the target T2I model, then use a VQA model to classify the attribute in each image and score bias severity via normalized entropy of class distributions, avoiding per-attribute classifiers.
Key results
- LLaVA-1.5-13B is the best VQA model on FairFace-labelled COCO generated images (gender acc 92.83, age acc 72.27, race acc 55.91).
- OpenBias agrees with human judgement with AME 0.15 on bias intensity and 67% majority-class agreement across 2200 user responses.
- OpenBias finds novel biases such as train color, laptop brand, horse breed and bed type on Stable Diffusion XL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Llava1.5-13B as the VQA model in OpenBias | OpenBias · COCO generated images | accuracy and F1 | Helped |
| Compare OpenBias severity scores with human judgments | OpenBias context-aware evaluation · User study | absolute mean error: AME = 0.15 | Helped |
| Use OpenBias to identify the majority class | OpenBias context-aware evaluation · User study | majority-class agreement: 67% | Helped |
Surprising findings
- Context-aware and context-free bias scores are not consistently correlated; e.g., motorcycle type is strongly biased only within context.
Limitations
- Inherits biases of the LLM (LLaMA-2) and VQA (LLaVA) models used for proposal and assessment.
- Context-aware vs context-free analysis is only qualitative.
- Socially sensitive attributes such as gender are treated as closed sets for evaluation against FairFace.
Experimental setup
Datasets: Flickr30k COCO FairFace
Benchmarks: FairFace (gender, age, race agreement) human user study
Metrics: accuracy F1 KL divergence absolute mean error majority-class agreement
Baselines: FairFace classifier (closed-set reference) CLIP-L OFA-Large mPLUG-Large BLIP-Large Llava1.5-7B
Builds on: in-context learning TIFA FairDiffusion
Closest papers by meaning
- FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models (NeurIPS 2025 · similarity 0.88)
- On Fairness of Unified Multimodal Large Language Model for Image Generation (NeurIPS 2025 · similarity 0.87)
- Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusion (CVPR 2025 · similarity 0.87)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.87)
- Implicit Bias Injection Attacks against Text-to-Image Diffusion Models (CVPR 2025 · similarity 0.86)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.