Research map / Image generation / Text-to-image diffusion
TIIF-Bench: How Does Your T2I Model Follow Your Instructions?
Xinyu Wei, Jinrui Zhang, Zeqing Wang, Hongyang Wei, Zhen Guo, Bairui Li, Lei Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Qwen3VL-8B with direct binary prediction vs off-the-shelf (ablation) | TIIF Evaluator (Qwen3VL-8B) · evaluator test set (560 questions) | accuracy: 84.11 -> 89.46 | Improved |
| use representation-based classification head instead of direct prediction (ablation) | TIIF Evaluator (Qwen3VL-8B) · evaluator test set | accuracy: 89.46 -> 91.43 | Improved |
| use reasoning-based (CoT) prediction instead of direct prediction (ablation) | TIIF Evaluator (Qwen3VL-8B) · evaluator test set | accuracy: 89.46 -> 91.07 | Improved |
| reasoning-based vs representation-based paradigm (ablation) | TIIF Evaluator (Qwen3VL-8B) · evaluator test set | accuracy: 91.43 vs 91.07 | No clear effect |
| use long instead of short prompts | weaker T2I models · TIIF-Bench overall | overall score: SD 3 66.5 -> 67.0; DALL-E 3 72.2 -> 70.2; MidJourney v7 67.7 -> 62.7 | Mixed |
| scale up and use closed-source models | T2I models · TIIF-Bench overall (short) | overall score: NB-Pro 93.1 vs Qwen-Image 87.0 | Improved |
| unified AR model Janus-Pro vs diffusion PixArt-Sigma | T2I models · TIIF-Bench overall (short) | overall score: 61.0 -> 65.5 | Improved |
Closest papers by meaning
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.91)
- Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) (NeurIPS 2024 · similarity 0.91)
- DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models (NeurIPS 2026 · similarity 0.90)
- Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist Rewards (NeurIPS 2026 · similarity 0.90)
- FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges (NeurIPS 2025 · similarity 0.89)
- Revisiting text-to-image evaluation with Gecko: on metrics, prompts, and human rating (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
