Research map / Image generation / Text-to-image diffusion
Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
Yushi Hu, Reyhane Askari-Hemmat, Melissa Hall, Emily Dinan, Luke Zettlemoyer, Marjan Ghazvininejad
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| preference-train Qwen2.5-VL-7B as EditReward | Qwen2.5-VL-7B · MMRB2 image editing (single-image subset) | accuracy: 57.8 -> 67.2 | Improved |
| preference-train Qwen2.5-VL-7B as UnifiedReward | Qwen2.5-VL-7B · MMRB2 text-to-image | accuracy: 50.4 -> 59.8 | Improved |
| preference-train Qwen2.5-VL-7B as UnifiedReward | Qwen2.5-VL-7B · MMRB2 multimodal reasoning (text-only-output subset) | accuracy: 53.7 -> 55.1 | Improved |
| add preference training to CLIPScore (ImageReward) | CLIPScore · MMRB2 text-to-image | accuracy: 51.0 -> 54.0 | Improved |
| add preference training to CLIPScore (HPSv2) | CLIPScore · MMRB2 text-to-image | accuracy: 51.0 -> 54.7 | Improved |
| add preference training to CLIPScore (PickScore) | CLIPScore · MMRB2 text-to-image | accuracy: 51.0 -> 58.6 | Improved |
| use larger open MLLM judge Qwen3-VL-32B instead of preference-trained HPSv3 on text-to-image | Qwen3-VL-32B · MMRB2 text-to-image | accuracy: 60.2 -> 64.1 | Improved |
| scale Gemma 3 judge from 4B to 27B | Gemma 3 · MMRB2 (average of 4 tasks) | accuracy: 50.7 -> 57.3 | Improved |
| scale Qwen2.5-VL judge from 7B to 72B | Qwen2.5-VL · MMRB2 (average of 4 tasks) | accuracy: 50.9 -> 59.0 | Improved |
| evaluate different-model pairs instead of same-model pairs (ablation) | Gemini 3 Pro judge · MMRB2 image generation | accuracy: 70.4 -> 79.7 | Improved |
| majority vote over K=9 samples instead of K=1 (ablation) | Gemini 2.5 Pro judge · MMRB2 (average of 4 tasks) | accuracy: 71.3 -> 72.5 | Improved |
| majority vote over more samples K (ablation) | Qwen3-VL judges · MMRB2 (average of 4 tasks) | accuracy | No clear effect |
Closest papers by meaning
- BaseReward: A Strong Baseline for Multimodal Reward Model (ICLR 2026 · similarity 0.93)
- ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoning (CVPR 2026 · similarity 0.91)
- Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences (ICLR 2026 · similarity 0.91)
- MM-RLHF: The Next Step Forward in Multimodal LLM Alignment (ICML 2025 · similarity 0.90)
- Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluators (NeurIPS 2025 · similarity 0.90)
- The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
