Research map / Vision-language and multimodal / Multimodal LLMs

SCoRe: Salience-Coverage Reduction for Vision Token Pruning in Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SCoRe, a training-free greedy visual token pruning method inside the vision encoder that jointly scores salience ([CLS] attention) and coverage (cosine distance to selected set), motivated by weighted k-center.

Problem

Visual token sequences in LVLMs are long, making self-attention costly; existing encoder-side pruning uses a decoupled Top-k-then-merge heuristic that suffers sampling collapse and loses secondary semantic clusters.

Approach

Formalize visual token pruning as a weighted k-center representativeness problem and greedily select, at each step, the token maximizing (min cosine distance to selected set) times (salience^alpha), unifying salience and coverage in one score.

Key results

• On LLaVA-1.5-7B retaining 32 of 576 tokens (94.4% pruning), SCoRe keeps 95.0% of full performance versus 91.5% for VisPruner and 77.9% for SparseVLM.
• On LLaVA-NeXT-7B retaining 160 of 2880 tokens, SCoRe reaches 93.4% relative performance versus 86.7% for VisPruner.
• On LLaVA-1.5-7B retaining 128 tokens, SCoRe achieves 100.6% relative performance, matching the full-token upper bound.

Surprising findings

• Pure diversity (alpha=0) and pure salience (large alpha) ablations each underperform the balanced score, showing salience-only Top-k is not sufficient.

Limitations

• Evaluated only on a few LVLM families (LLaVA series, Qwen-VL, Video-LLaVA) and relies on [CLS] attention as salience, which may not transfer to encoders without a CLS token.
• Hyperparameter alpha is tuned on TextVQA at 32 tokens; the sensitivity study covers only one benchmark.
• Efficiency comparison is limited to FastV on LLaVA-NeXT and POPE.

Experimental setup

Datasets: VQAv2 GQA VizWiz ScienceQA-IMG TextVQA POPE MME MMBench MMBench-CN MM-Vet TGIF-QA MSVD-QA

Benchmarks: VQAv2 GQA VizWiz ScienceQA-IMG TextVQA POPE MME MMBench MMBench-CN MM-Vet TGIF-QA MSVD-QA

Metrics: accuracy relative performance retention score

Baselines: FastV SparseVLM ToMe VisionZip VisPruner

Builds on: Weighted k-Center Problem VisionZip VisPruner FastV

Closest papers by meaning

• Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.94)
• LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.92)
• ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.92)
• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.92)
• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.92)
• IF-Prune: Information-Flow Guided Token Pruning for Efficient Vision-Language Models (CVPR 2026 · similarity 0.92)