Research map / Vision-language and multimodal / Multimodal LLMs
LLaVA-Critic: Learning to Evaluate Multimodal Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds LLaVA-Critic-113k, a GPT-4o-generated critic instruction dataset, and fine-tunes LLaVA-OneVision 7B/72B into an open generalist multimodal evaluator that also supplies reward signals for iterative DPO.
Problem
Open-source large multimodal models lack reliable generalist evaluators that can score or rank model responses across diverse multimodal tasks with reasoning, as a free alternative to GPT-4V/4o judges, and reward signals for preference learning currently depend on human feedback or costly proprietar
Approach
Curate a critic instruction-following dataset (pointwise scoring with evaluation prompts from GPT-as-a-judge benchmarks and pairwise ranking with 30 prompt templates), labeled with GPT-4o judgments and reasons, then fine-tune LLaVA-OneVision so it outputs scores or preferences plus justifications, and use its pairwise scores to build self-generated preference data for iterative DPO.
Key results
- LLaVA-Critic-72B reaches average pointwise Pearson-r 0.754 with GPT-4o versus 0.634 for LLaVA-OV-72B, and LLaVA-Critic-7B reaches 0.732.
- On MLLM-as-a-Judge, LLaVA-Critic-72B obtains 0.393 score correlation with humans (vs. 0.314 for LLaVA-Critic-7B) and 71.5% pairwise accuracy without ties.
- Iterative DPO with LLaVA-Critic rewards improves LLaVA-OV-7B on LLaVA-W from 90.7 to 100.3 and WildVision from 54.0 to 67.3 after 3 rounds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train LLaVA-OV-7B on LLaVA-Critic-113k (full data) instead of baseline | LLaVA-OV-7B as judge · In-domain pointwise scoring (7 benchmarks) | Pearson-r avg: 0.364 -> 0.732 | Helped |
| Scale critic model from 7B to 72B base | LLaVA-Critic · In-domain pointwise scoring (7 benchmarks) | Pearson-r avg: 0.732 -> 0.754 | Helped |
| Use LLaVA-Critic as reward for iterative DPO on LLaVA-v1.5-7B (9.4k prompts) vs RLAIF-V and other self-reward methods | LLaVA-v1.5-7B · Preference learning, 5 benchmarks | LLaVA-W / WildVision / MMHal: LLaVA-W 73.5 vs 72.7 (RLAIF-V); WildVision 29.2 vs 20.4 base; MMHal 2.07 vs 3.04 (RLAIF-V) | Mixed |
| Replace LLaVA-RLHF human-trained reward with LLaVA-Critic-7B reward in iterative DPO | LLaVA-OV-7B · Visual chat (LLaVA-W, L-Wilder, WildVision, LiveBench, MMHal, VideoDC) | LLaVA-W: 97.5 (LLaVA-RLHF) -> 100.3 | Helped |
| Replace LLaVA-RLHF reward with LLaVA-Critic-72B reward in iterative DPO | LLaVA-OV-72B · Visual chat benchmarks | LLaVA-W: 103.2 -> 104.4 | Helped |
| Use zero-shot LLaVA-OV-7B as reward model instead of LLaVA-Critic in DPO (ablation) | LLaVA-OV-7B · Visual chat (WildVision) | WildVision: 67.3 (Critic-7B) vs 66.6 (OV-7B reward) | Hurt |
| Iterative DPO rounds with LLaVA-Critic-7B reward on OV-7B (ablation) | LLaVA-OV-7B · LLaVA-W | LLaVA-W: 90.7 -> 96.7 (iter-1) -> 97.0 (iter-2) -> 100.3 (iter-3) | Helped |
| Use best-of-5 selection with LLaVA-Critic-7B at inference (ablation) | LLaVA-OV-7B after 3-round DPO · LLaVA-W / L-Wilder | LLaVA-W, L-Wilder: LLaVA-W 100.3 -> 102.0; L-Wilder 71.6 -> 74.8 | Helped |
| Add pairwise critic data to pointwise-only training (ablation) | LLaVA-Critic-7B data curation · MLLM-as-a-Judge | Score Pearson: 0.311 (pointwise only) vs 0.314 (all) | No clear effect |
| Train on pairwise data only instead of combined data (ablation) | LLaVA-Critic-7B data curation · MLLM-as-a-Judge | Score Pearson: 0.314 -> 0.249 | Hurt |
| Add tie pairs to pairwise training data (ablation) | LLaVA-Critic-7B data curation · MLLM-as-a-Judge | Pair w. Tie: 0.552 -> 0.556 (w. Tie); 0.683 -> 0.689 (w.o. Tie) | Helped |
| Train on LLaVA-Critic 53k subset (v0.5) instead of full 113k data (ablation) | LLaVA-Critic-7B · In-domain pointwise scoring | Pearson-r avg: 0.732 -> 0.712 | Hurt |
Surprising findings
- LLaVA-Critic-7B with the 9.4k-prompt DPO outperforms the LLaVA-RLHF human-feedback reward model and matches GPT-4o-based iterative DPO on several visual chat benchmarks.
- Pointwise-only critic data improves pairwise ranking more than pairwise-only data does (0.543 vs 0.541 tie accuracy; score 0.311 vs 0.249).
Limitations
- Pairwise in-domain evaluation uses only 2k sampled pairs from WildVision Arena, and MLLM-as-a-Judge results were re-implemented because the full pipeline was not released.
- Training labels come from GPT-4o judgments, so the critic inherits GPT-4o biases and may not match human judgment in all settings.
- Pairwise accuracy with ties and Kendall's tau for LLaVA-Critic-72B still trail GPT-4V/4o slightly.
Experimental setup
Datasets: LLaVA-Instruction-150k SVIT ComVint LLaVAR LRV-Instruction M3IT LLaVA-Med PCA-EVAL VLFeedback RLHF (LLaVA-RLHF) RLHF-V WildVision Arena battle data
Benchmarks: LLaVA-in-the-Wild LLaVA-Wilder ImageDC MM-Vet WildVision-Bench MMHal-Bench MLLM-as-a-Judge WildVision Arena pairwise battles LiveBench Video Detailed Captioning MME MMBench
Metrics: Pearson correlation Kendall's tau pairwise accuracy (with and without ties) win/score on visual chat benchmarks
Baselines: GPT-4o GPT-4V Prometheus-Vision LLaVA-RLHF reward model LLaVA-OneVision zero-shot judge RLAIF-V CSR SIMA
Builds on: LLaVA-OneVision DPO VLFeedback LLaVA-RLHF
Closest papers by meaning
- Improved Baselines with Visual Instruction Tuning (CVPR 2024 · similarity 0.90)
- Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Following (CVPR 2026 · similarity 0.89)
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities (ICML 2024 · similarity 0.88)
- MM-RLHF: The Next Step Forward in Multimodal LLM Alignment (ICML 2025 · similarity 0.88)
- Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs (NeurIPS 2024 · similarity 0.88)
- Beyond task performance: evaluating and reducing the flaws of large multimodal models with in-context-learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.