Research map / Image generation / Text-to-image diffusion
DSH-Bench: A Difficulty- and Scenario-Aware Benchmark with Hierarchical Subject Taxonomy for Subject-Driven Text-to-Image Generation
Zhenyu Hu, Qing Wang, Cao Te, Kuo Liao, Longfei Lu, Liqun Liu, Shuang Li, Hang Chen, Mengge Xue, Yuan Chen, Chao Deng, Peng Shu and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fine-tuned SICS metric instead of GPT-4o judge for subject preservation | human alignment · 1,600 held-out image pairs | Kendall tau: 0.619 -> 0.677 | Improved |
| use SICS instead of GPT-4o | human alignment · MS-Diffusion / OmniGen (Spearman) | Spearman: OmniGen 0.697 vs 0.667 | Mixed |
| evaluate on hierarchical-taxonomy DSH-Bench instead of DreamBench | subject-driven T2I models · subject preservation | score: BLIP-Diffusion 0.229 -> 0.204 | Hurt |
| increase subject difficulty from easy to hard | subject-driven T2I models · DSH-Bench subject preservation | score | Hurt |
| increase subject difficulty from easy to hard | subject-driven T2I models · DSH-Bench prompt following | CLIP-T | No clear effect |
| move from BC to IE prompt scenario | subject-driven T2I models · DSH-Bench | all dimensions | Hurt |
| use larger newer backbone (closed-source Nano-Banana vs open models) | subject-driven T2I models · DSH-Bench leaderboard | S_h: 0.256 -> 0.272 | Improved |
| trade subject preservation for prompt following | subject-driven T2I models · DSH-Bench | SP vs PF | Mixed |
Closest papers by meaning
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.88)
- DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models (NeurIPS 2026 · similarity 0.88)
- Revisiting text-to-image evaluation with Gecko: on metrics, prompts, and human rating (ICLR 2025 · similarity 0.87)
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.87)
- A High Quality Dataset and Reliable Evaluation for Interleaved Image-Text Generation (ICLR 2026 · similarity 0.87)
- ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
