Research map / Vision-language and multimodal / Multimodal LLMs

VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VLM-Pruner, a training-free visual token pruning method that selects tokens near-to-far from pivots using a Buffering for Spatial Sparsity criterion, then merges discarded tokens via similarity-weighted aggregation.

Problem

Visual tokens in VLMs cause large compute costs; existing training-free pruning either ignores inter-token redundancy or yields spatially dispersed selections that miss object details.

Approach

A centrifugal near-to-far selection: max-min pivots, then a spatial-distance-modulated similarity (BSS) that defers spatially distant candidates, parallel greedy selection with an annealed threshold, and similarity-weighted aggregation of discarded tokens into retained ones.

Key results

• At 88.9% pruning on LLaVA-1.5-7B (64 tokens), VLM-Pruner retains 95.61% of upper-bound average, versus 93.68% for DivPrune and 92.71% for DART.
• On LLaVA-1.5-13B at 64 tokens, it reaches 92.68% average, exceeding DivPrune by 2.48 points.
• On Qwen2-VL-7B at 88.9% pruning, it attains 91.20%/96.57% style average gains; OCRBench score 581 vs 481 for DART, with Avg improvement of +3.65% reported in text.

Surprising findings

• DivPrune collapses to 46.47% average when substituted as the stage-2 selector in the ablation on LLaVA-1.5-7B.
• BTP retains only 62.06% average on Qwen3-VL-4B at 88.9% pruning, far below other methods.

Limitations

• Pruning is applied at a single decoder layer (layer 2) with a fixed spatial grid assumption; generalization to non-grid token layouts is not shown.
• Some reported numbers are inconsistent across text and tables (e.g., ablation averages of 95.30% vs. 95.61% and a 46.47% DivPrune ablation result).

Experimental setup

Datasets: GQA MMBench MME POPE ScienceQA TextVQA OCRBench SEED-Bench OK-VQA EgoSchema NExTQA VideoMME

Benchmarks: GQA MMBench MME POPE ScienceQA TextVQA OCRBench SEED-Bench OK-VQA EgoSchema NExTQA VideoMME

Metrics: accuracy average retained performance (%) relative to upper bound total inference time speedup FLOPs

Baselines: FastV SparseVLM PDrop MustDrop DART DivPrune CDPruner SAINT BTP

Builds on: FastV DART DivPrune

Closest papers by meaning

• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.93)
• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.92)
• Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning (ICLR 2026 · similarity 0.92)
• IF-Prune: Information-Flow Guided Token Pruning for Efficient Vision-Language Models (CVPR 2026 · similarity 0.92)
• IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruning (ICLR 2026 · similarity 0.92)
• ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.91)