AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models

Jaemin Cho, Abhay Zala, Mohit Bansal

ICCV 2023 · Image generation › Text-to-image diffusion · approach: T2I evaluation and controllable adapters · 298 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
compare Stable Diffusion with minDALL-E on counting and spatial skillstext-to-image models finetuned on PaintSkills · PaintSkillsaccuracy (%): count 37.8 vs 47.5; spatial 7.9 vs 50.7; object 96.2 vs 89.9Mixed
model generation vs oracle upper bound on count and spatialT2I models · PaintSkillsaccuracy (%): oracle 97.8/96.2 vs best 47.5/50.7Hurt
reduce finetuning data from 100% to 50% (ablation)minDALL-E and Stable Diffusion · PaintSkillsavg accuracy (%): 62.7 -> 64.3 (minDALL-E); 47.3 -> 48.6 (SD)No clear effect
reduce finetuning data from 100% to 10% (ablation)minDALL-E · PaintSkills spatialaccuracy (%): 50.7 -> 38.2Hurt
use BLIP-2 instead of CLIP for gender detection (ablation)bias evaluation pipeline · COCO bias test and Adienceaccuracy (%): 66% -> 82%Improved
use BLIP-2 instead of CLIP for attribute detection (ablation)bias evaluation pipeline · attribute detectionaccuracy (%): 79% -> 92%Improved
compare gender MAD across modelsT2I models · gender-neutral promptsMAD: minDALL-E 0.1984 vs Karlo 0.3545 vs SD 0.3618Mixed
compare skin tone MAD across modelsT2I models · diagnostic promptsMAD: 0.1687 / 0.1707 / 0.1698No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy