Research map / Vision-language and multimodal / Multimodal LLMs

Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PruneSID, a training-free visual token compression method for VLMs that groups tokens via PCA over the token dimension (PSCA), prunes redundancy with intra-group NMS, and adapts per-image token budgets by an information score.

Problem

Visual tokens in VLMs are excessive and redundant, and existing compression methods struggle to balance preserving important tokens with keeping information diverse at high compression ratios.

Approach

Cluster tokens into semantic groups using low-rank PCA applied to the token dimension (PSCA), then run NMS within each group using an image-level redundancy score to set the similarity threshold, plus an information-aware per-image token budget.

Key results

• On LLaVA-1.5 with 64 tokens (11.1% retention, 576 base), PruneSID-Dyn reaches 96.3% of vanilla average accuracy, vs. VisionZip at 94.0%.
• On LLaVA-NeXT with 160 tokens (5.6% retention), PruneSID averages 92.8% of vanilla, vs. VisionZip's 90.3%, and prefilling drops from 218ms to 27.8ms (7.8x).
• On Video-LLaVA with 6.6% token retention, PruneSID averages 95.5% vs. VisionZip's 93.2%.

Surprising findings

• Retaining the least important tokens first (Descend) causes severe degradation (84.2% at 64 tokens), confirming importance matters.
• Qwen2.5-VL results show PruneSID trailing or matching VisionZip on some benchmarks (e.g., MME 2218 vs 2317 at 33.3% retention).

Limitations

• Fine-grained, instruction-specific queries can lose relevant tokens under extreme compression (task-agnostic design).
• Gains over baselines are often small on some benchmarks and models (e.g., Qwen2.5-VL MME, LLaVA-1.5 some benchmarks).
• Theoretical lower bound relies on an assumed redundancy/information decomposition and is loosely justified.

Experimental setup

Datasets: GQA MMBench MME POPE ScienceQA VQA-v2 TextVQA MMMU SEED-Bench VizWiz LLaVA-Bench TGIF

Benchmarks: GQA MMBench MME POPE ScienceQA VQA-v2 TextVQA MMMU SEED-Bench VizWiz LLaVA-Bench TGIF

Metrics: accuracy relative performance vs. vanilla F1 prefilling time inference time

Baselines: FastV SparseVLM MustDrop DART ToMe LLaVa-PruMerge FasterVLM VisPruner HiRED DivPrune VisionZip PACT

Builds on: VisionZip DART DivPrune FastV SparseVLM

Closest papers by meaning

• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.92)
• IF-Prune: Information-Flow Guided Token Pruning for Efficient Vision-Language Models (CVPR 2026 · similarity 0.92)
• Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.92)
• DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference (CVPR 2026 · similarity 0.91)
• SCoRe: Salience-Coverage Reduction for Vision Token Pruning in Vision-Language Models (CVPR 2026 · similarity 0.91)
• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.91)