Research map / Vision-language and multimodal / Multimodal LLMs

Look-Back: Implicit Visual Re-focusing in MLLM Reasoning

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Look-Back, a two-stage SFT-then-GRPO method that uses a <back> token and a format reward to make Qwen2.5-VL-7B spontaneously re-focus on the image during reasoning, without re-injecting visual inputs.

Problem

MLLMs increasingly rely on text and lose visual attention in later reasoning stages; prior fixes explicitly re-inject images or visual tokens. The paper asks whether MLLMs can learn to re-focus on visual input implicitly and on their own.

Approach

Prompt-level analysis shows that a <back> token prompt makes the model re-attend to image tokens during later reasoning. Look-Back cold-starts with SFT on GPT-o4-mini-inserted <back> verification segments, then runs GRPO with only a format reward that favors <back> outputs, so visual re-focusing emerges without explicit image re-input or architecture changes.

Key results

• Solution-back-7B reaches 56.4 average on math benchmarks vs 48.5 for the Qwen2.5-VL-7B base, and 67.3 overall average, comparable to VL-Rethinker-7B (66.0) and above NoisyRollout-7B (65.8).
• Semantic-back-7B achieves 67.6 overall average and 79.6 perception average, the best overall among compared 7B reasoning models in the table.
• Ablation: removing RL drops Semantic-back overall average from 67.6 to 58.5; removing SFT drops it to 66.7.

Surprising findings

• Prompting alone triggers <back> in only 62.48% of responses, and the paper reports that Qwen2-VL trained with this format reward learned reward hacking by emitting empty <back></back> segments.
• Cold-start SFT on GPT-4o-generated CoT with inserted <back> tokens degraded performance; switching to self-generated model outputs improved it.

Limitations

• Reward hacking on weaker base models (Qwen2-VL) with empty <back> segments; method relies on Qwen2.5-VL's pretrained reflection ability.
• Cold-start SFT on purely math data reduced perception scores slightly as data scale grew.
• Comparisons with other reasoning MLLMs use differing training data and durations, making direct comparison difficult.

Experimental setup

Datasets: Geo170K MathV360K Geometry3K K12 Geometry3K validation set

Benchmarks: MathVerse MathVision MathVista WeMath GeoMath HallusionBench TallyQA MME

Metrics: accuracy average accuracy per category trigger rate attention ratio

Baselines: GRPO VL-Rethinker-7B NoisyRollout-7B Vision-R1-7B OpenVLThinker-7B MM-Eureka-Qwen-7B GPT-4o Qwen2.5-VL-32B

Builds on: GRPO Qwen2.5-VL DeepSeek-R1 LLaMA-Factory EasyR1

Closest papers by meaning

• Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.92)
• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.92)
• When to Think and When to Look: Uncertainty-Guided Lookback (CVPR 2026 · similarity 0.92)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.92)
• Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning (ICML 2026 · similarity 0.92)
• GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking (CVPR 2026 · similarity 0.91)