Research map / Vision-language and multimodal / Multimodal LLMs

VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VisionPulse, a training-free step-wise visual token pruning method for multimodal reasoning that re-selects visual tokens at each decoding step using a visual attention mass budget, retaining ~5% of visual tokens while shortening reasoning traces and keeping accuracy.

Problem

Static visual token pruning at prefill assumes the needed visual evidence stays fixed during long multimodal chain-of-thought decoding, which causes accuracy loss and longer reasoning traces under aggressive retention.

Approach

Recompute visual token importance (FastV-style attention from the current query at an anchor layer) at every decoding step, and set the per-step retention budget from a temperature-scaled visual attention mass that correlates strongly with the number of activated visual tokens.

Key results

• At 5% visual token retention per step, FastV-VP on Qwen3-VL-Thinking-4B keeps average accuracy at 71.35 vs 72.68 full-token baseline (-1.8%) while reducing generation length by 11.2%.
• At 10% retention, FastV-VP matches full-token average accuracy (72.69 vs 72.68) with 12.3% shorter reasoning traces, whereas static baselines lose 39.7-47.6% accuracy.
• Up to 1.30x end-to-end latency speedup over dense attention in the reported setting (batch size 8, 1k generated tokens).

Surprising findings

• Retaining redundant visual context throughout decoding can steer the model toward query-irrelevant cues and lengthen reasoning traces, so pruning can shorten reasoning.
• Static pruning methods such as LOOK-M increase average generation length by up to 108% under aggressive retention while losing accuracy.

Limitations

• Evaluated mainly on one model family (Qwen3-VL-Thinking 4B) with limited generalization checks on two other backbones.
• Temperature and budget clamping hyperparameters (tau per retention setting, clamped K_t ranges) are tuned per setting, and the 5% and 10% comparisons use clamped budgets for fairness.
• Uses an LLM judge (GPT-4o / GPT-4o-mini) for several benchmarks, and the paper does not report code release.

Experimental setup

Datasets: CharXiv RQ InfoVQA ChartQA MMStar RealWorldQA MMVet MIA-Bench

Benchmarks: CharXiv RQ InfoVQA ChartQA MMStar RealWorldQA MMVet MIA-Bench

Metrics: accuracy average generation length end-to-end latency

Baselines: VisionZip FastV LOOK-M PDrop

Builds on: FastV VisionZip LOOK-M PDrop

Closest papers by meaning

• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.92)
• Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning (ICML 2026 · similarity 0.91)
• Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings (NeurIPS 2025 · similarity 0.91)
• See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs (CVPR 2026 · similarity 0.91)
• SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.91)
• UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs (NeurIPS 2026 · similarity 0.91)