Research map / Vision-language and multimodal / Multimodal LLMs

Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AdaViG, a training-free gate that aborts harmful self-generated visual steps in unified multimodal models early in diffusion, using attention-based Generation Intent and Visual Fidelity signals to improve accuracy and reduce visual-generation cost on math reasoning.

Problem

Interleaved reasoning in unified multimodal models (UMMs) can produce visual steps that are harmful to final accuracy, and frequent visual generation adds substantial latency and KV-cache memory overhead.

Approach

Two early-diffusion attention statistics (Generation Intent: text-vs-input-image attention balance; Visual Fidelity: attention to input ViT tokens) predict whether an image generation will help; the gate aborts the visual step at step 5 of 50 only when both signals are weak.

Key results

• AdaViG on Bagel-Canvas improves DynaMath accuracy by up to 5.7 points (42.3 to 48.0) while cutting image-generation FLOPs by 75%.
• On Bagel-Zebra-CoT, AdaViG raises average accuracy from 43.9 to 47.0 and reduces end-to-end latency by up to 45.6% on WeMath.
• Gating with both signals (AdaViG) outperforms single-signal and OR-combined gates, and random or text-only aborting, on Bagel-Canvas.

Surprising findings

• About 26% of self-generated visual steps are harmful on MathVision, comparable to the 28% that are helpful, so expected utility of intrinsic image generation is near zero.
• Random aborting of image generations and text-only reasoning both perform worse than the unmodified Bagel-Canvas baseline on several benchmarks, yet the signal-based gate improves it.

Limitations

• Gate decides whether to generate a visual aid but does not improve visual token quality; benefits limited when visual generation is rarely triggered.
• Thresholds tuned on MathVision testmini, and the same benchmark family is used for evaluation, so tuning generalization is uncertain.
• Evaluated only on Bagel-based UMMs with math benchmarks.

Experimental setup

Datasets: MathVision testmini (threshold tuning)

Benchmarks: MathVista testmini (GPS) MathVerse testmini OlympiadBench (Math) WeMath DynaMath

Metrics: accuracy image-generation FLOPs reduction wall-clock latency

Baselines: GPT-4o Gemini-2.5-Pro Qwen2.5-VL-7B InternVL3-8B Emu3-Chat-8B Janus-Pro-7B MUSE-VL-7B Bagel-7B CoT ICoT DAP-ICoT MathCanvas

Builds on: MathCanvas Bagel-Zebra-CoT DAP-ICoT Bagel

Closest papers by meaning

• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.89)
• Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.89)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.89)
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (ICML 2026 · similarity 0.89)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.89)
• Latent Implicit Visual Reasoning (CVPR 2026 · similarity 0.89)