Research map / Image generation / Text-to-image diffusion
Beyond Aesthetics: Cultural Competence in Text-to-Image Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CUBE, a benchmark of ~300K KB+LLM-grounded cultural artifacts and 1K prompts across 8 countries and 3 concepts, evaluating text-to-image models on cultural awareness (human-rated faithfulness/realism) and cultural diversity (quality-weighted Vendi score).
Problem
Existing text-to-image benchmarks evaluate faithfulness, realism, and aesthetics but not cultural competence, i.e., whether models accurately depict culture-specific artifacts (awareness) and produce diverse rather than stereotyped cultural artifacts for under-specified prompts (diversity).
Approach
Ground cultural concepts in a knowledge graph (Wikidata traversal) refined by GPT-4 Turbo to build a large artifact concept space, derive a popularity-filtered prompt set for human cultural-awareness evaluation, and measure cultural diversity with a quality-weighted Vendi score using geographic similarity kernels.
Key results
- Imagen 2 and SDXL both score below 3 on average for faithfulness and realism in many country-concept cells, with lower scores for Brazil, Turkey, and Nigeria, especially for faithfulness.
- Cultural diversity scores are low across all models; the best normalized qVS with uniform kernel weights is 0.19 for Imagen 2 on cuisine, far from the maximum of 1.
- Faithfulness and realism correlate positively (mean rho 0.400 for Imagen 2), while diversity correlates weakly with faithfulness (rho 0.016) and realism (rho 0.156).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Imagen 2 instead of SDXL | T2I generation on CUBE-1K cuisine prompts · Cuisine, across eight countries | faithfulness: Imagen generally outperforms SDXL; SDXL scores higher for USA | Mixed |
| Use Imagen 2 rather than SDXL | T2I generation on CUBE cultural prompts · Cultural awareness, across concepts and countries | faithfulness and realism | Mixed |
| Use the uniform similarity kernel instead of a single-level kernel | CUBE cultural diversity evaluation · Cuisine, Landmarks, and Art | qVS | Mixed |
Surprising findings
- Higher faithfulness and realism for cultural prompts do not translate into higher cultural diversity of generated artifacts.
- Raters found Imagen 2 cultural relevance high (e.g., 98.4% for India) but lower for Turkey (68.1%) and Nigeria (73.6%).
Limitations
- Covers only 8 countries and 3 concepts, and the country selection may be biased.
- Inter-annotator agreement for faithfulness and realism is low to moderate (Krippendorff's alpha often below 0.3 for several countries).
- Mapping generated images to artifacts is only about 70% accurate on human-validated samples.
Experimental setup
Datasets: Wikidata Google Images reference set Google Search popularity
Benchmarks: CUBE-1K CUBE-CSpace
Metrics: faithfulness (1-5 human Likert) realism (1-5 human Likert) cultural relevance quality-weighted Vendi score (qVS) HPSv2 quality
Baselines: Imagen 2 Stable Diffusion XL Playground Realistic Vision
Builds on: Vendi score quality-weighted Vendi score Wikidata HPSv2
Closest papers by meaning
- Culture In a Frame: C$^3$B as a Comic-Based Benchmark for Multimodal Culturally Awareness (ICLR 2026 · similarity 0.90)
- All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages (CVPR 2025 · similarity 0.89)
- No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Culture Affordance Atlas: Reconciling Object Diversity Through Functional Mapping (AAAI 2026 · similarity 0.88)
- Multilingual Diversity Improves Vision-Language Representations (NeurIPS 2024 · similarity 0.87)
- See It from My Perspective: How Language Affects Cultural Bias in Image Understanding (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.