Research map / Vision-language and multimodal / Multimodal LLMs

Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VRCD, a training-free inference-time reranking that penalizes confidence of masked tokens whose token-to-image attention overlaps with other high-confidence candidates, reducing visual redundancy in parallel decoding of dMLLMs.

Problem

Confidence-based parallel decoding in diffusion multimodal LLMs ranks masked positions independently and can commit tokens in the same step that rely on overlapping visual grounding, leaving less complementary context for later steps.

Approach

Defines a Visual Redundancy Index (VRI) over tokens committed in parallel and a training-free reranker that reweights each candidate's confidence by a redundancy score computed from pairwise overlap of visually salient token-to-image attention within a confidence-truncated candidate window.

Key results

• On M3CoT at length 384, VRCD reaches 41.99 vs 38.35 for confidence at FR 0.5 (about 9.5% relative gain); gains up to 18.8% relative on M3CoT are reported in longer decoding.
• Mean VRI at L=192, FR=0.5 drops from 0.685 (LaViDa) to 0.588 (VRCD).
• VRCD adds about 1.5% runtime overhead (46.264 vs 46.982 tokens/s) on M3CoT, while IG drops to 24.884 tokens/s.

Surprising findings

• Minimizing visual redundancy alone can degrade decoding quality, so the method keeps confidence as the primary signal and uses a moderate alpha.
• IG, an information-gain method, yields almost no VRI reduction relative to LaViDa baseline.

Limitations

• Does not adapt the decoding schedule or per-step commit size; only reranks candidates within the schedule.
• Attention-based overlap may be noisy for format tokens and function words; alpha and lambda chosen as fixed values rather than tuned per setting.
• Evaluated mainly on a single dMLLM backbone (LaViDa) with a supplementary MMaDA check.

Experimental setup

Datasets: M3CoT MMBench ScienceQA DocVQA InfoVQA

Benchmarks: M3CoT MMBench SQA-IMG DocVQA InfoVQA

Metrics: accuracy ANLS VRI remaining-position entropy tokens/s position-change rate

Baselines: Confidence Margin Entropy InfoGain

Builds on: LLaDA LaViDa InfoGain

Closest papers by meaning

• Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings (NeurIPS 2025 · similarity 0.90)
• Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models (ICML 2026 · similarity 0.90)
• Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders (ICLR 2026 · similarity 0.89)
• Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding (CVPR 2026 · similarity 0.89)
• Q Cache: Visual Attention Is Valuable in Less than Half of Decode Layers for Multimodal Large Language Model (AAAI 2026 · similarity 0.89)
• Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference (AAAI 2025 · similarity 0.89)