Research map / Image generation / Text-to-image diffusion

T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation

CVPR 2025 · Image generation › Text-to-image diffusion · approach: Fairness and bias in generative models · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces T2ISafety, a hierarchical benchmark (12 tasks, 44 categories) with 70K collected prompts and 68K human-annotated images, plus ImageGuard, an InternLM-XComposer2-based MLLM evaluator, and a normalized KL-divergence fairness metric, used to evaluate 12-15 T2I models.

Problem

Existing T2I safety benchmarks cover only narrow slices of toxicity, fairness, and privacy, lack quality-controlled prompts and large annotated data, and rely on weak automated evaluators (CLIP-based or pretrained MLLMs) that miss risks.

Approach

Combine a three-domain safety taxonomy with quality-controlled prompts and a large human-labeled image set to train a dedicated MLLM safety evaluator (ImageGuard) using cross-modality attention and a contrastive loss, and measure fairness with a normalized KL divergence.

Key results

• ImageGuard achieves F1 of 0.869 (fairness), 0.779 (toxicity), and 0.875 (privacy) on its own test set, outperforming GPT-4o (0.470 toxicity, 0.356 privacy).
• ImageGuard reaches F1 of 0.689 on UnsafeDiff, 0.704 on SMID, and 0.683 on UnsafeBench, above the best compared baselines on those OOD sets.
• SDXL achieves the highest average toxicity safety rate (0.826) and privacy safety rate (0.672) among 12 evaluated T2I models, while racial NKL-Div remains high across most models.

Measured effects reported in the paper

Surprising findings

• Newer T2I models do not necessarily have better fairness, toxicity mitigation, or privacy protection than older versions.
• Proprietary MLLMs such as GPT-4o and Gemini1.5-pro perform poorly on the toxicity and privacy subsets, and refuse fairness judgments about gender, age and race.

Limitations

• Safety judgments are subjective and rely on category definitions from regulations and provider policies.
• Fairness uses only binary gender and four age and five race groups, a simplified coarse taxonomy.
• ImageGuard uses relatively low-resolution inputs from InternLM-XComposer2.

Experimental setup

Datasets: T2ISafety prompt set (70K prompts) T2ISafety image set (68K images) LAION2B-en FACET FairFace UnsafeDiff SMID UnsafeBench

Benchmarks: T2ISafety UnsafeDiff SMID UnsafeBench

Metrics: F1 score safety rate normalized KL divergence

Baselines: Idefics2 LlavaNext InternVL2 GPT-4o Claude3.5-sonnet Gemini1.5-pro LlavaGuard Multiheaded PerspectiveVision SD filter

Builds on: InternLM-XComposer2 LlamaGuard Q16 concept erasing (ESD)

Closest papers by meaning

• T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models (NeurIPS 2024 · similarity 0.91)
• OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models (NeurIPS 2025 · similarity 0.91)
• Position: Towards Implicit Prompt For Text-To-Image Models (ICML 2024 · similarity 0.90)
• Six-CD: Benchmarking Concept Removals for Text-to-image Diffusion Models (CVPR 2025 · similarity 0.89)
• Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts (ICML 2024 · similarity 0.89)
• Detect-and-Guide: Self-regulation of Diffusion Models for Safe Text-to-Image Generation via Guideline Token Optimization (CVPR 2025 · similarity 0.89)