Research map / Vision-language and multimodal / Multimodal LLMs

ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ST-Veto, a training-free decoding-time method for diffusion MLLMs that vetoes temporally unstable (second-order Taylor residual) and weakly visually grounded (image attention mass) tokens during unmasking and swaps in safe near-boundary candidates.

Problem

Diffusion multimodal LLMs (dMLLMs) are underexplored for long-horizon multimodal reasoning, and AR-style CoT methods transfer poorly to parallel diffusion decoding, where unstable or weakly grounded tokens get committed early.

Approach

Use the diffusion model's global visibility of all masked positions across steps: predict each token's confidence with a second-order Taylor expansion over a 4-step history and measure image-attention mass; veto top-k tokens that are unstable or ungrounded (MAD-based thresholds) and swap them with safe candidates near the confidence boundary.

Key results

• On LaViDa at L/T=256/128, ST-Veto reaches 38.25 on MMBench vs 31.55 for the base confidence decoder and 48.54 on SQA-IMG vs 40.06.
• On MMaDA at L/T=256/128, ST-Veto improves V* Bench from 31.41 to 37.17 and M3CoT from 25.11 to 27.74.
• Inference throughput of ST-Veto is nearly identical to the base decoder (e.g., 12.92 vs 12.88 samples/s on LaViDa 256/128), while AR-style baselines roughly double cost.

Surprising findings

• Adapted VLM reasoning methods (DDCoT, CCoT, ICoT, SCAFFOLD) often fail to improve or degrade performance when applied to dMLLMs.
• Standard remasking heuristics (confidence, margin, entropy) show nearly identical performance.

Limitations

• Relies on internal confidence and attention signals, which are imperfect proxies for correctness.
• Evaluated only on single-image reasoning with two dMLLMs; multi-image, video, and ultra-long generation are not studied.
• Gains are reported on benchmarks where base dMLLM performance is low, and the paper uses a manually chosen set of hyperparameters (MAD thresholds, 4-step history).

Experimental setup

Datasets: M3CoT ScienceQA MMBench V* Bench

Benchmarks: M3CoT MMBench SQA-IMG V* Bench MME POPE MMMU

Metrics: accuracy

Baselines: Base-Confidence (Low-conf) Base-Margin Base-Entropy DDCoT CCoT SCAFFOLD ICoT EAH OPERA FarSight

Builds on: LLaDA LaViDa MMaDA Chain-of-Thought prompting low-confidence remasking

Closest papers by meaning

• Thinking Diffusion: Penalize and Guide Visual-Grounded Reasoning in Diffusion Multimodal Language Models (CVPR 2026 · similarity 0.89)
• ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming (AAAI 2025 · similarity 0.88)
• Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference (AAAI 2025 · similarity 0.88)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.88)
• Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens (CVPR 2025 · similarity 0.88)
• Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.88)