Research map / Image generation / Text-to-image diffusion
WISE: World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation
Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Fanqing Meng, Kun-Peng Ning, Bin Zhu, Li Yuan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Rewrite WISE prompts into direct prompts with GPT-4o (ablation) | 20 T2I models · WISE | WiScore overall: Janus-Pro-1B +0.34; FLUX.1-dev 0.50 -> 0.73 | Improved |
| Rewrite prompts for physics prompts on MetaQuery-XL (ablation) | MetaQuery-XL · WISE Physics | WiScore: -0.04 | Hurt |
| Add chain-of-thought before generation (ablation) | BAGEL · WISE | WiScore overall: 0.52 -> 0.70 | Improved |
| Use unified MLLM with AR+Diffusion architecture instead of dedicated T2I | T2I models · WISE | WiScore overall: Qwen-Image 0.62 vs FLUX.1-dev 0.50 | Improved |
| Use purely autoregressive unified model | Janus-Pro / Emu3 / VILA-U · WISE | WiScore overall: Janus-1.3B 0.23, Janus-Pro-7B 0.35 | Hurt |
| Scale up AR model from Janus-Pro-1B to 7B (ablation) | Janus-Pro · WISE | WiScore overall: 0.26 -> 0.35 | Improved |
| Change WiScore weights to 0.5/0.3/0.2 or 0.4/0.4/0.2 (ablation) | WiScore metric · WISE (gemini judge) | model ranking: Spearman 0.993 | No clear effect |
| Swap GPT-4o judge for gemini-3.1-flash-lite or Qwen3.5-122B-A10B (ablation) | WiScore evaluation · WISE | model ranking | No clear effect |
| Vary random seed (42-46) (ablation) | representative T2I models · WISE | WiScore: Qwen-Image 0.5029 ± 0.0046 | No clear effect |
| Use recent AR models (NextFlow, Emu3.5) rather than early AR | AR unified models · WISE | WiScore: Emu3 0.39 -> NextFlow 0.62 | Improved |
| Rewrite prompts for physics (scientific categories) (ablation) | T2I models · WISE Physics | rewrite-ineffective ratio: Physics 85.1% | Mixed |
Closest papers by meaning
- MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning (NeurIPS 2025 · similarity 0.89)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.89)
- Beyond Aesthetics: Cultural Competence in Text-to-Image Models (NeurIPS 2024 · similarity 0.89)
- Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective (ICLR 2025 · similarity 0.88)
- Localizing and Editing Knowledge In Text-to-Image Generative Models (ICLR 2024 · similarity 0.88)
- DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
