Research map / Vision-language and multimodal / Multimodal LLMs

ReaGEN: Adaptive Generation of Structured Chains-of-Thought for Efficient Multimodal Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReaGEN, which uses teacher-guided evolutionary search with attention-derived stage importance to build query-specific CoT structures and trains a tiny 18M generator to predict these structures for a frozen Qwen3-VL-4B student.

Problem

LVLMs struggle with complex compositional visual reasoning; existing remedies are costly inference-time tree/multi-path search or expensive SFT/RL post-training on large CoT datasets.

Approach

Learn a lightweight generator that predicts sample-adaptive stage sequences (CoT structures) from image, question, and cross-stage attention summaries of a default CoT, with the VLM kept frozen; training data come from a teacher-guided evolutionary search whose rewards and stage-importance scores come from attention.

Key results

• On Qwen3-VL-4B, ReaGEN (4 iter) reaches 64.51% on VStar vs 49.27% direct... (up to +26 points over VReST on MathVision/MMStar-style gains, e.g. MathVision 75.77% vs 49.27% for VReST)
• ReaGEN reduces average inference tokens by 79% (about 4x fewer than VReST) and VLM calls by 53% on average.
• Math-domain results: ReaGEN (4 iter) is within 1 point of VReST on MathVision and MathVerse, while outperforming VReST on most vision benchmarks.

Surprising findings

• GEN trained on MathVision transfers to vision benchmarks such as MMStar, gaining up to +26 points over VReST, while MMStar-trained GEN slightly hurts math benchmarks.
• Removing attention-derived signals consistently degrades performance across benchmarks.

Limitations

• Requires open-weight VLM access to internal attention signals, so not applicable to closed-source models.
• Gap to teacher-guided search upper bound remains large on MathVision (ReaGEN 44.60% vs T-Search 74.03% in ablation).
• Math-centric gains are only on par with VReST rather than superior.

Experimental setup

Datasets: MMMU MathVerse (text-only) MathVision MMStar

Benchmarks: MMMU-Pro (10-option) MMMU-Pro (4-option) VStar MMStar MathVision MathVerse (vision-only)

Metrics: accuracy token usage number of VLM calls

Baselines: Direct Answer CoT VReST AStar Socratic MCTS GPT-4o GPT-4o-mini InternVL3.5-4B Qwen3-VL-8B

Builds on: Chain-of-Thought prompting VReST AStar Socratic-MCTS Multimodal Chain-of-Thought

Closest papers by meaning

• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.91)
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.91)
• HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning (NeurIPS 2026 · similarity 0.91)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.91)
• VGR: Visual Grounded Reasoning (ICLR 2026 · similarity 0.90)
• MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO (NeurIPS 2025 · similarity 0.90)