Research map / Image generation / Text-to-image diffusion
GenColorBench: A Color Evaluation Benchmark for Text-to-Image Generation
Muhammad Atif Butt, Alexandra Gomez-Villa, Tao Wu, Javier Vazquez-Corral, Joost Van De Weijer, Kai Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compare numerical color prompts (RGB/hex) with color names | T2I models · GenColorBench | accuracy: Numerical 3.66-17.49 vs Name 29.55-49.85 (BLIP3o 28.31) | Hurt |
| increase task complexity from single object to multi-object composition | T2I models · GenColorBench | accuracy: Multi-Object scores 1.73 to 11.43 | Hurt |
| use a multimodal architecture (BLIP3o) for numeric colors | BLIP3o · Numerical Color Understanding | accuracy: 28.31 vs next best 17.49 | Improved |
| object categories with intrinsic color (animals, fruits and vegetables) vs decorative color (clothes, vehicles, furniture) | T2I models · GenColorBench color name | accuracy | Hurt |
| intermediate colors instead of basic colors | T2I models · Color Name Accuracy | accuracy: accuracy drops by 8--20% | Hurt |
| -ish modifiers vs light/dark modifiers | T2I models · ISCC-NBS L3 color name | accuracy: -ish often below 35% | Hurt |
| light modifiers vs dark modifiers | T2I models · ISCC-NBS L3 color name | accuracy | Improved |
| higher resolution (1024 vs 512) | OmniGen2 vs SD 3.5/BLIP3o · GenColorBench | average accuracy: 22.42 (512) vs 21.80 / 22.21 (1024) | No clear effect |
| use VLM-based VQA for color evaluation instead of pixel-level metric | Qwen2-VL-7B vs authors' metric · Synthetic Blender images | accuracy: 49.01 -> 96.46 (L2 binary vs ours) | Hurt |
Closest papers by meaning
- ColorConceptBench: A Benchmark for Probabilistic Color-Concept Understanding in Text-to-Image Models (NeurIPS 2026 · similarity 0.89)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.88)
- TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation (ICML 2025 · similarity 0.87)
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.87)
- Free-Lunch Color-Texture Disentanglement for Stylized Image Generation (NeurIPS 2025 · similarity 0.87)
- ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
