Research map / Vision-language and multimodal / Multimodal LLMs

IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IVC-Prune, a training-free prompt-aware visual token pruning method that keeps RoPE-derived implicit visual coordinate tokens plus value-similarity-selected foreground tokens, retaining about 50% of visual tokens while preserving roughly 99%+ performance.

Problem

Existing visual token pruning methods for LVLMs focus on semantic relevance and discard spatially critical tokens, causing large drops on visual grounding and spatial reasoning tasks under high-resolution inputs.

Approach

Analyzes RoPE rotation matrices to find token positions whose rotations approximate identity (real axis, via cosine sum) or 90-degree rotation (imaginary axis, via sine sum), treating them as implicit visual coordinates to preserve; foreground tokens are chosen by a two-stage value-vector similarity procedure (semantic seeds then contextual refinement) to avoid positional bias of attention scores.

Key results

• On RefCOCO-family grounding, IVC-Prune retains 99.6% (Qwen2.5-VL-7B), 99.5% (InternVL 2.5-8B) and 99.0% (DeepSeek-VL2) of vanilla performance at about 50% visual tokens.
• On general VQA, IVC-Prune reaches 100.6%, 99.6%, 100.1% and 101.3% relative average across Qwen2.5-VL-7B, InternVL 2.5-8B, DeepSeek-VL2 and LLaVA-v1.5 respectively.
• Removing IVC tokens from IVC-Prune drops RefCOCO testA from 92.0 to 76.0, while adding 10% IVC tokens to FastV raises it from 74.4 to 82.1.

Surprising findings

• Early-layer pruning sensitivity reported in prior work stems mainly from removing IVC tokens rather than pruning itself.
• Text-image attention-based foreground selection drops RefCOCO testA by 12 points versus value-similarity selection on InternVL 2.5.

Limitations

• Fixed pruning ratios not adapted to task or visual complexity.
• Pruning layer chosen from a small validation subset rather than automatically.
• IVC identification is tied to RoPE and may not transfer to learned or other positional encodings; OCRBench recognition tasks degrade substantially (66.3 vs 88.4).

Experimental setup

Datasets: RefCOCO RefCOCO+ RefCOCOg SEEDBench MMBench MMStar MME POPE HallusionBench RealWorldQA TextVQA AI2D

Benchmarks: RefCOCO RefCOCO+ RefCOCOg SEEDBench MMBench MMStar MME POPE HallusionBench RealWorldQA TextVQA AI2D

Metrics: accuracy relative average performance KV cache size prefill time decode latency total time

Baselines: FastV Window FastV PDrop VScan SparseVLM

Builds on: RoPE FastV PDrop VLMEvalKit

Closest papers by meaning

• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.92)
• VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm (CVPR 2026 · similarity 0.92)
• Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning (ICLR 2026 · similarity 0.91)
• SwiftVLM: Efficient Vision-Language Model Inference via Cross-Layer Token Bypass (NeurIPS 2026 · similarity 0.91)
• IF-Prune: Information-Flow Guided Token Pruning for Efficient Vision-Language Models (CVPR 2026 · similarity 0.91)
• LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.90)