Research map / Vision-language and multimodal / Multimodal LLMs

Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VISTA, a vision-aware self-improvement framework for MLLMs that combines prefix resampling (reusing correct reasoning prefixes via critical-token detection) with a vision-aware attention score to filter hallucinated correct solutions, improving SFT and DPO/GRPO post-training.

Problem

Self-improvement training of multimodal LLMs suffers from data imbalance (few correct solutions for hard queries) and language prior bias (correct-answer solutions that hallucinate or ignore visual cues), with only answer correctness as a quality signal.

Approach

Resample new solutions from the correct prefix of failed solutions, located via a self-calibration token-swap test (critical tokens outside the model's Top-5), and filter correct solutions using a vision-aware attention score measuring attention mass on visual tokens in a middle layer.

Key results

• VISTA-SFT gives +13.66% average gain over SFT-Seed for Qwen2.5-VL-3B-Instruct (65.87 vs 52.21 overall) after 3 iterations.
• On Qwen2.5-VL-7B, VISTA-SFT reaches 68.87 overall, +6.67 over SFT-Seed.
• VISTA-DPO reaches 65.12 overall on Qwen2.5-VL-3B, +1.98 over IRPO (63.14).

Surprising findings

• Some self-improvement baselines (e.g., ReST-EM, R3V) show model collapse over iterations, especially on Geometry3K.
• Training on the lowest-VAS solutions causes catastrophic degradation, while Answer PPL and Reasoning Depth metrics can underperform random selection.

Limitations

• Experiments limited to small models (3B and 7B); larger or MoE models untested.
• Prefix resampling may reduce trajectory diversity when critical tokens appear late, risking overfitting.
• VAS filtering relies on a hand-set threshold tau=-0.5 and middle-layer choice.

Experimental setup

Datasets: SLAKE VQA-Rad Geometry3K ScienceQA ChartQA

Benchmarks: SLAKE VQA-Rad Geometry3K ScienceQA ChartQA PathVQA MathVista IllusionBench MMMU MathVerse BLINK-Twice

Metrics: zero-shot accuracy average accuracy gain over SFT-Seed

Baselines: SFT-Seed SFT-Oracle STaR ReST-EM RFT R3V IRPO STaR+

Builds on: STaR ReST-EM R3V IRPO DPO GRPO

Closest papers by meaning

• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.91)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.91)
• Sherlock: Self-Correcting Reasoning in Vision-Language Models (NeurIPS 2025 · similarity 0.91)
• SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement (NeurIPS 2025 · similarity 0.91)
• Diving into Self-Evolving Training for Multimodal Reasoning (ICML 2025 · similarity 0.91)
• Look-Back: Implicit Visual Re-focusing in MLLM Reasoning (AAAI 2026 · similarity 0.91)