Research map / Vision-language and multimodal / Multimodal LLMs

Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning

CVPR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Critic-V, a Reasoner-Critic framework in which a DPO-trained Qwen2-VL-7B critic gives natural-language critiques that iteratively refine a VLM's answers, with preference pairs built by error insertion and a Jaccard-based rule reward.

Problem

VLMs often produce inaccurate or irrelevant answers from hallucinated image understanding and unrefined reasoning, and existing self-correction methods rely on the model's own abilities without external, reliable feedback.

Approach

Decouple reasoning from critique: a separate critic VLM, trained with DPO on critique preference pairs built by GPT-4o-inserted visual errors and ranked by a Jaccard-plus-GPT rule-based reward, gives natural-language feedback that the reasoner uses to revise its prompt and answer at inference time.

Key results

• Qwen2-VL-7B+Critic-V reaches 73.2 on MathVista (+11.8 over 61.4) and 32.9 on MathVerse (+7.1), with the highest score on five of eight benchmarks.
• LLaVA-v1.5-7B+Critic-V improves RealWorldQA from 50.7 to 63.5 (+12.8) and MMT-Bench from 36.0 to 47.4 (+11.4).
• On MMT-Bench, Qwen2-VL-7B with Self-Refine drops to 57.8 versus 62.0 with Critic-V, showing the DPO-trained critic outperforms untrained self-refinement.

Measured effects reported in the paper

Surprising findings

• Qwen2-VL-7B+Critic-V drops on MMStar (56.2 vs 60.7 baseline), a regression on one benchmark.
• Self-Refine with Qwen2-VL-7B lowers MMT-Bench accuracy (57.8 vs 60.4 baseline).

Limitations

• Critic training data is synthetic, generated by GPT-4o error insertion and scored with a GPT-4o-based term, so it may inherit GPT-4o biases.
• Evaluation relies on GPT-4o-based scoring terms and a single reasoner-critic iteration setup with limited ablations.
• Gains are uneven across benchmarks, with a decline on MMStar for Qwen2-VL-7B.

Experimental setup

Datasets: critique-VQA

Benchmarks: RealWorldQA MMStar MMBench SEEDBench ScienceQA MMT-Bench MathVista MathVerse

Metrics: accuracy (%)

Baselines: GPT-4V GeminiPro-Vision Llama-3.2-11B-Vision MiniCPM-V 2.6 InternVL2-8B POVID CSR SIMA SCL Self-Refine

Builds on: Direct Preference Optimization TextGrad Self-Refine CriticGPT In-Context Reinforcement Learning

Closest papers by meaning

• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
• Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training (ICML 2026 · similarity 0.90)
• VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use (ICLR 2026 · similarity 0.90)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.90)
• See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection (CVPR 2026 · similarity 0.90)