Research map / Vision-language and multimodal / Multimodal LLMs
IVTP: Instruction-guided Visual Token Pruning for Large Vision-Language Models
Kai Huang, Hao Zou, Ye Xi, Bochen Wang, Zhen Xie, Liang Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use attention rollout instead of plain current-layer CLS attention for token importance (ablation) | IVTP GTP module · 12 benchmarks avg | average accuracy (retrained): 58.1 -> 62.0 | Improved |
| prune per group of layers with rollout instead of layer-wise (ablation) | IVTP · 12 benchmarks avg | average accuracy (retrained): 59.5 -> 62.0 | Improved |
| aggregate intra-group attention with residual rollout rather than mean, max or multiply (ablation) | IVTP · 12 benchmarks avg | average accuracy (retrained): 60.7 (best alternative, max) -> 62.0 | Improved |
| add instruction-guided pseudo CLS token in LLM stage (ablation) | IVTP · 12 benchmarks avg | average accuracy (retrained): 59.2 -> 62.0 | Improved |
| use CLIP text encoder to pick key text tokens for the pseudo CLS (ablation) | IVTP · 12 benchmarks avg | average accuracy (retrained): 59.9 -> 62.0 | Improved |
| extend visual-only pruning methods (TopK, ToMe) into the LLM (ablation) | TopK and ToMe · 12 benchmarks avg | average accuracy (retrained): 60.0 -> 60.1 (ToMe) | No clear effect |
| add LLM-stage pruning to ViT-only IVTP (ablation) | IVTP-V · 12 benchmarks avg | average accuracy (retrained): 60.8 -> 62.0 | Improved |
| group 3 layers per pruning group versus 2, 4, 6 or all (ablation) | IVTP · 12 benchmarks avg | average accuracy (retrained): 58.2 (ALL) -> 62.0 (3 layers) | Improved |
| reduce visual tokens from 576 to 256-512 with IVTP | LLaVA-1.5-7B · 12 benchmarks avg | average accuracy | No clear effect |
| IVTP versus other pruning methods at 64 tokens | LLaVA-1.5-7B · 12 benchmarks avg | average accuracy: 60.0 (ToMe) -> 62.0 | Improved |
Closest papers by meaning
- ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.95)
- LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.95)
- QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.94)
- Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.94)
- ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning (AAAI 2026 · similarity 0.94)
- IF-Prune: Information-Flow Guided Token Pruning for Efficient Vision-Language Models (CVPR 2026 · similarity 0.94)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
