Research map / Vision-language and multimodal / Multimodal LLMs
Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLMs
Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| visual instruction tuning of the LLM into an MLLM (compare text-only description input) (ablation) | Qwen2.5-VL-7B-Instruct vs Qwen2.5-7B-Instruct · Creation-MMBench-TO | Reward: -19.18 -> -27.50 | Hurt |
| visual instruction tuning of the LLM into an MLLM (ablation) | MiniCPM-o-2.6 vs Qwen2.5-7B-Instruct · Creation-MMBench-TO | Reward: -19.18 -> -36.57 | Hurt |
| visual instruction tuning of the LLM into an MLLM (ablation) | InternVL2.5-8B vs InternLM2.5-7B-Chat · Creation-MMBench-TO | Reward: -22.19 -> -28.73 | Hurt |
| give GPT-4o text descriptions instead of images (ablation) | GPT-4o-1120 · Creation-MMBench | Reward: 0.36 -> 6.96 | Improved |
| remove visual input from MLLMs (ablation) | Gemini-1.5-Pro-002, Qwen2-VL-72B, MiniCPM-o-2.6 · Creation-MMBench | Reward | Hurt |
| use dual evaluation (swap response positions) instead of single evaluation (ablation) | MLLM-as-a-judge · Judge alignment with human preference | consistency (%): 88.85 -> 92.13 (GPT-4o judge) | Improved |
| higher OpenVLM objective score does not imply higher creativity reward | InternVL2.5-78B-MPO · Creation-MMBench vs OpenVLM | OC Score vs Reward: OC 77.0, Reward -12.55 (vs GPT-4o 72.0, 0.00) | Mixed |
| use preference optimization (MPO) for InternVL2.5 8B | InternVL2.5-8B · Creation-MMBench | Reward: -25.42 -> -15.10 | Improved |
Closest papers by meaning
- CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product (AAAI 2026 · similarity 0.91)
- IDEA-Bench: How Far are Generative Models from Professional Designing? (CVPR 2025 · similarity 0.90)
- MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models (NeurIPS 2025 · similarity 0.89)
- CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions (CVPR 2026 · similarity 0.86)
- Thinking with Programming Vision: Towards a Unified View for Thinking with Images (CVPR 2026 · similarity 0.86)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
