AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models

Jiacheng Ruan, Wenzhen Yuan, Xian Gao, Ye Guo, Daoxin Zhang, Zhe Xu, Yao Hu, Ting Liu, Yuzhuo Fu

ICCV 2025 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 19 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase model size from small group to middle groupopen-source LVLMs as VLRMs · VLRMBench step-basedW-Avg F1: 29.8 -> 45.9Improved
increase model size from middle group to large groupopen-source LVLMs as VLRMs · VLRMBench step-basedW-Avg F1: 45.9 -> 46.1No clear effect
add confident wording (CM) to erroneous stepssmall-group VLRMs · VLRMBench SC vs CMF1: 6.8% degradationHurt
add hallucinated information (EH task) to reasoningsmall-group VLRMs · VLRMBench SC vs EHF1: 14.6% degradationHurt
add hallucinated attributes (AH task) to reasoningsmall-group VLRMs · VLRMBench SC vs AHF1: 14.1% degradationHurt
swap numerical errors (DE) for general errors (SC)large-group VLRMs · VLRMBench DE vs SCF1: decreased by 2.5%No clear effect
use spatial relationship errors (SR) instead of SC errorslarge-group VLRMs · VLRMBench SR vs SCF1: dropped by 30.4%Hurt
use image reference errors (IC) instead of SC errorslarge-group VLRMs · VLRMBench IC vs SCF1: dropped by 15.8%Hurt
place better reasoning second instead of first (ablation)Llama3.2-11B · VLRMBench MJaccuracy: 52.8 -> 8.4Hurt
increase reasoning length from <=10 to >15 stepsGPT-4o · VLRMBench SCF1: 77.8 -> 68.6Hurt
add one-shot in-context example (ablation)Llama3.2-11B · VLRMBench SCF1: 33.9 -> 42.2Improved
add one-shot in-context example (ablation)Qwen2.5VL-3B · VLRMBench SCF1: 29.9 -> 31.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy