Research map / Vision-language and multimodal / Multimodal LLMs

Perception in Reflection

ICML 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reflective Perception (RePer), a policy-critic multi-turn refinement loop for LVLMs, trained via Reflective Perceptual Learning with reflective unlikelihood training on a self-generated visual reflection dataset built from LLaVA-665K images.

Problem

Large vision-language models often produce hallucinated or incomplete perceptions in a single pass and cannot correct them; existing visual CoT approaches are task-specific and do not allow iterative correction.

Approach

Make perception iterative: a policy LVLM and a critic model alternate over multiple rounds, with the policy trained on multi-turn reflective dialogues ordered from low- to high-reward responses, using a weighted likelihood/unlikelihood loss whose weights follow rewards.

Key results

• RePer on LLaVA-1.5 13B reaches 54.73 CAPTURE on DetailCaps-4870 vs 51.23 for the base model, and 51.00 on HallusionBench aAcc vs 43.85.
• RePer on LLaVA-1.5 7B raises DetailCaps CAPTURE from 51.03 to 52.89 and GAPE total from 75.16 to 80.88.
• In an ablation on DetailCaps, RPL outperforms DPO and LiPO on the LLaVA-1.5 13B base (54.73 vs 50.53 and 52.31 CAPTURE).

Measured effects reported in the paper

Surprising findings

• Reflection gains without RPL are smaller at early turns, and LLaVA-1.5 alone often fails to correct hallucinations when given critic feedback (case study).

Limitations

• Visual reflection dataset is small (about 11k samples) and built from a single source image set.
• Relies on GPT-4o and Gemini-based scoring and proprietary judges, so reward and evaluation depend on closed models.
• Reflection turns increase inference cost; paper does not quantify latency.

Experimental setup

Datasets: LLaVA-665K ShareGPT4V-102k DetailCaps HallusionBench MMHal-Bench GAIVE LLaVABench

Benchmarks: LLaVABench HallusionBench MMHal-Bench GAIVE DetailCaps-4870 GAPE

Metrics: CAPTURE accuracy hallucination rate GPT-4o rating score CLIP/DINO/DINOv2 image similarity

Baselines: MiniGPT-4 mPLUG-Owl InstructBLIP LLaVA-SFT+ LLaVA-RLHF VOLCANO LLaVA-1.5 DPO LiPO

Builds on: LLaVA-1.5 SCoRe RISE DPO LiPO

Closest papers by meaning

• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.88)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.87)
• In-Context Learning Can Help Vision Language Models Overcome Training Prior (NeurIPS 2026 · similarity 0.87)
• BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception (NeurIPS 2025 · similarity 0.86)
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging (ICML 2025 · similarity 0.86)
• Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning (ICML 2026 · similarity 0.86)