Research map / Vision-language and multimodal / Multimodal LLMs
Sherlock: Self-Correcting Reasoning in Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Sherlock trains Llama3.2-Vision-11B reasoning VLMs to self-correct via a trajectory-level preference objective, visual-perturbation negatives, and a dynamic beta, then self-improves online without labels.
Problem
Reasoning VLMs cannot effectively self-correct errors in their multi-step reasoning, and existing improvement methods need large annotated data or accurate verifiers that limit generalization.
Approach
Trajectory-level self-correction objective that revises only the erroneous suffix of a reasoning trace, with negatives built by truncating reasoning and injecting visual noise, a sample-specific dynamic beta, and online self-improvement using self-consistency-filtered self-corrections as preferences.
Key results
- Sherlock Iter2 reaches 64.1 average accuracy with direct generation and 65.4 after three rounds of self-correction across eight benchmarks, versus LLaVA-CoT 63.2, Mulberry 63.9, LlamaV-o1 63.4.
- Existing reasoning VLMs (LLaVA-CoT, Mulberry, LlamaV-o1) do not improve or degrade under self-correction prompts; LlamaV-o1 drops from 63.4 to 48.2.
- On MathVista, sequential self-correction with MM-Verify reaches 55.9 accuracy at 8.7 A100 GPU hours versus 55.1 for parallel voting at 40.2 GPU hours.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply three rounds of self-correction | Sherlock Iter2 · 8 benchmarks | average accuracy: 64.1 -> 65.4 | Helped |
| add one round of online self-improvement training | Sherlock Offline · 8 benchmarks | average accuracy: 63.2 -> 63.9 | Helped |
| add a second round of online self-improvement training | Sherlock Iter1 · 8 benchmarks | average accuracy: 63.9 -> 64.1 | Helped |
| apply three rounds of self-correction | LLaVA-CoT · 8 benchmarks | average accuracy: 63.2 -> 63.0 | Hurt |
| apply three rounds of self-correction | Mulberry · 8 benchmarks | average accuracy: 63.9 -> 63.8 | Hurt |
| apply three rounds of self-correction | LlamaV-o1 · 8 benchmarks | average accuracy: 63.4 -> 48.2 | Hurt |
| add LLaVA-Critic external critiques | Sherlock Iter2 · 8 benchmarks | average accuracy: 64.1 -> 62.9 | Hurt |
| add Critic-V external critiques | Sherlock Iter2 · 8 benchmarks | average accuracy: 64.1 -> 63.2 | Hurt |
| use Qwen2.5-VL-7B external critiques | Sherlock Iter2 · 8 benchmarks | average accuracy: 64.1 -> 64.9 | Helped |
| use verifier-guided sequential self-correction instead of unverified self-correction | Sherlock Iter2 · MathVista | accuracy: 54.0 -> 55.9 | Helped |
| use Sherlock-SFT instead of standard SFT (ablation) | LLaVA-CoT 20k training comparison · 8 benchmarks | average accuracy: 62.0 -> 62.8 | Helped |
| add three rounds of self-correction | Sherlock SFT 20k · 8 benchmarks | average accuracy: 62.8 -> 63.4 | Helped |
Surprising findings
- Self-correction prompts degrade LLaVA-CoT and Mulberry and sharply degrade LlamaV-o1 (63.4 to 48.2), indicating existing reasoning VLMs cannot self-correct.
- Adding a self-correction loss alone improves direct reasoning accuracy, suggesting self-correction and reasoning are mutually reinforcing.
Limitations
- Self-correction gains depend on external critic quality for critique-based correction, and weaker critics degrade results.
- Evaluation relies on the 20k LLaVA-CoT subset and Llama3.2-Vision-11B only; generalization to other base models is not shown.
- Self-correction costs extra inference rounds (three rounds), increasing inference compute.
Experimental setup
Datasets: LLaVA-CoT
Benchmarks: MMBench-V1.1 MMVet HallusionBench MMMU MMStar AI2D MathVista MME
Metrics: accuracy average accuracy
Baselines: LLaVA-CoT Mulberry LlamaV-o1 LLaVA-Critic Critic-V Qwen2.5-VL-7B-Instruct Majority Vote
Builds on: DPO SRPO ARIES LLaVA-CoT MM-Verify
Closest papers by meaning
- Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training (ICML 2026 · similarity 0.91)
- VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning (CVPR 2025 · similarity 0.90)
- Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs (ICLR 2026 · similarity 0.90)
- Learning Self-Correction in Vision–Language Models via Rollout Augmentation (ICML 2026 · similarity 0.90)
- Enhancing Large Vision Language Models with Self-Training on Image Comprehension (NeurIPS 2024 · similarity 0.90)
- Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.