Research map / Vision-language and multimodal / Multimodal LLMs

Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Chain-of-Evidence (CoE) for VD-RAG, where CoT steps are grounded to bounding boxes and page indices, and trains Qwen2.5-VL-7B with SFT cold start followed by GRPO using stepwise attribution, grounding, accuracy, and format rewards (LAT).

Problem

Visual evidence attribution in visual document RAG lacks fine-grained, step-level traceability: existing methods link only the final answer to a source region and require large annotated datasets.

Approach

Interleave CoT reasoning with bounding-box and page-index evidence per reasoning step (CoE), and train with GRPO using a stepwise reward that scores CRAG-style image-text alignment of each cropped evidence region (via ColQwen2) with its reasoning step, gated by answer correctness and penalized for overlapping boxes.

Key results

• LAT-Ind. improves vanilla Qwen2.5-VL-7B by +7.95% average EM and +44.6% average IoU@0.5 across single- and multi-image settings (paper text).
• On Paper->Wiki transfer (single), LAT-Ind. reaches 67.7 EM and 35.6 IoU@0.5 vs SFT 66.0 and 29.4.
• LAT-Full achieves 41.4 IoU@0.5 and 75.2 SA on Wiki-VISA multi-image, versus 31.6 IoU@0.5 for VISA-7B.

Surprising findings

• Removing the answer-correctness gate from the step reward causes significant drops in both accuracy and grounding, showing process and outcome rewards must be coupled.
• One-shot in-context demonstration yields only a modest SA gain, while RL gives a large SA increase.

Limitations

• Trained only on VISA, whose evidence is predominantly single-hop, so multi-hop generalization is untested.
• Reward thresholds (tau, delta, epsilon) are set manually from annotated examples rather than adaptively.
• Cold-start data relies on Gemini 2.5 Pro generations with manual correction, and only about 30% retained.

Experimental setup

Datasets: Wiki-VISA Paper-VISA FineWeb-VISA

Benchmarks: Paper-VISA Wiki-VISA

Metrics: soft EM IoU@0.5 Stepwise Attribution (SA)

Baselines: VISA-7B Qwen2.5-VL-7B Qwen2.5-VL-32B Gemini2.0 flash Qwen-VL max LLaVA-CoT R1-OneVision InternVL2.5 SFT

Builds on: GRPO DeepSeek-R1 VISA ColQwen2 Qwen2.5-VL

Closest papers by meaning

• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.91)
• VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.89)
• DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding (CVPR 2026 · similarity 0.89)
• Benchmarking Retrieval-Augmented Multimomal Generation for Document Question Answering (NeurIPS 2025 · similarity 0.89)
• Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning (NeurIPS 2025 · similarity 0.89)
• Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs (ICLR 2026 · similarity 0.89)