AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Creation-MMBench: Assessing Context-Aware Creative Intelligence in MLLMs

Xinyu Fang, Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang, Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen and 1 more

ICCV 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
visual instruction tuning of the LLM into an MLLM (compare text-only description input) (ablation)Qwen2.5-VL-7B-Instruct vs Qwen2.5-7B-Instruct · Creation-MMBench-TOReward: -19.18 -> -27.50Hurt
visual instruction tuning of the LLM into an MLLM (ablation)MiniCPM-o-2.6 vs Qwen2.5-7B-Instruct · Creation-MMBench-TOReward: -19.18 -> -36.57Hurt
visual instruction tuning of the LLM into an MLLM (ablation)InternVL2.5-8B vs InternLM2.5-7B-Chat · Creation-MMBench-TOReward: -22.19 -> -28.73Hurt
give GPT-4o text descriptions instead of images (ablation)GPT-4o-1120 · Creation-MMBenchReward: 0.36 -> 6.96Improved
remove visual input from MLLMs (ablation)Gemini-1.5-Pro-002, Qwen2-VL-72B, MiniCPM-o-2.6 · Creation-MMBenchRewardHurt
use dual evaluation (swap response positions) instead of single evaluation (ablation)MLLM-as-a-judge · Judge alignment with human preferenceconsistency (%): 88.85 -> 92.13 (GPT-4o judge)Improved
higher OpenVLM objective score does not imply higher creativity rewardInternVL2.5-78B-MPO · Creation-MMBench vs OpenVLMOC Score vs Reward: OC 77.0, Reward -12.55 (vs GPT-4o 72.0, 0.00)Mixed
use preference optimization (MPO) for InternVL2.5 8BInternVL2.5-8B · Creation-MMBenchReward: -25.42 -> -15.10Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy