Research map / Vision-language and multimodal / Multimodal LLMs

TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TwinPrune, a spatial-density token score reused for bipartite-merge visual token pruning at prefill and top-k visual KV eviction at decode, and argues prefill-probe benchmarks cannot measure decode-phase compression.

Problem

Visual token pruning in VLMs is validated mostly on single-token benchmarks (MMBench, POPE, ScienceQA) that cannot detect decode-phase KV eviction effects; both prefill compute and decode KV memory need efficient handling.

Approach

A single training-free spatial density signal (average local cosine dissimilarity on the visual token grid) drives both a density-tilted farthest-point bipartite merge during prefill and one-shot top-k KV eviction during decode, plus a diagnostic split of benchmarks into prefill probes and full-pipeline probes.

Key results

• On InternVL-3.5-8B at 12.5% visual KV keep, density top-k KV eviction beats H2O by 9.6 pp on TextVQA.
• At L=0 prune 80%, TwinPrune reaches Avg-4 0.8003 and 3.24x prefill speedup on InternVL-3.5-8B, while DivPrune and DART collapse to near 0 accuracy.
• Removing all visual KV at decode leaves prefill-probe benchmarks nearly unchanged (MMBench mismatch 0.54%) but flips 45.5% of TextVQA predictions.

Surprising findings

• Pure-selection methods DivPrune and DART collapse to near-zero accuracy under aggressive prefill pruning on InternVL-3.5-8B, while merge-based density does not.
• Prefill-probe benchmarks such as MMBench, POPE, and ScienceQA are structurally insensitive to decode-phase KV eviction, so prior decode-phase claims validated only on them measure prefill quality.

Limitations

• Single-stage prefill compression; multi-stage scheduling unexplored.
• Spatial-grid density assumes regular tokenization, so non-grid tokenizers such as Q-Former need alternative formulations.

Experimental setup

Datasets: TextVQA MM-Vet DocVQA MMBench POPE ScienceQA V*Bench

Benchmarks: MMBench POPE ScienceQA TextVQA MM-Vet DocVQA V*Bench

Metrics: accuracy ANLS Avg-4 accuracy prefill speedup prediction mismatch

Baselines: Random ToMe FastV PruMerge DivPrune DART H2O SnapKV StreamingLLM

Builds on: DivPrune ToMe H2O SnapKV StreamingLLM

Closest papers by meaning

• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.90)
• Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.90)
• When Token Pruning is Worse than Random: Understanding Visual Token Information in VLLMs (CVPR 2026 · similarity 0.89)
• DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference (CVPR 2026 · similarity 0.89)
• Hi-Lo Prune: Look at What You'll Lose before Pruning with Hierarchical Token Selection (CVPR 2026 · similarity 0.89)
• Each Complexity Deserves a Pruning Policy (NeurIPS 2025 · similarity 0.89)