Research map / Vision-language and multimodal / Multimodal LLMs
PosPrune: Visual Token Pruning with Positional Bias Correction for Efficient Large Vision-Language Models
Ziyang Wang, Mengwei Li, Hao Yin, Wenhao Liu, Zilei Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PosPrune, a visual token pruning method for LVLMs that corrects positional bias in token selection to reduce computation.
Problem
Large numbers of visual tokens from vision encoders add substantial computational overhead to large vision-language models; existing pruning methods rely on [CLS]-based attention in the vision encoder or prune inside LLM decoding layers.
Approach
Prunes visual tokens while correcting positional bias in the selection signal, rather than relying on global [CLS]-attention selection or in-LLM pruning. Only the abstract is available, so the precise mechanism is not described.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ARP instead of global [CLS] selection at 352 tokens (ablation) | PosPrune (LLaVA-1.5-7B) · MME | score: 1687 -> 1751 | Improved |
| use ARP instead of global [CLS] selection at 352 tokens (ablation) | PosPrune (LLaVA-1.5-7B) · ScienceQA | accuracy: 67.4 -> 67.7 | Improved |
| add MMR and prune to 72 tokens (ablation) | PosPrune (LLaVA-1.5-7B) · MME | score: 1751 -> 1746 | No clear effect |
| add Positional Bias Correction (ablation) | PosPrune (LLaVA-1.5-7B) · MME | score: 1746 -> 1792 | Improved |
| add Positional Bias Correction (ablation) | PosPrune (LLaVA-1.5-7B) · ScienceQA | accuracy: 67.8 -> 68.2 | Improved |
| use polynomial degree 3 instead of degree 1 for bias fit (ablation) | PosPrune (LLaVA-1.5-7B) · MME | score: 1765 -> 1792 | Improved |
| use polynomial degree 3 instead of degree 5 for bias fit (ablation) | PosPrune (LLaVA-1.5-7B) · MME | score: 1663 -> 1792 | Improved |
| use degree 3 instead of degree 2 polynomial (ablation) | PosPrune (LLaVA-1.5-7B) · ScienceQA | accuracy: 68.1 -> 68.2 | No clear effect |
| use MMR with lambda 0.9 instead of relevance-only (lambda 1) (ablation) | PosPrune (LLaVA-1.5-7B) · MME and ScienceQA | score | Improved |
| fit bias with 800 samples instead of fewer (ablation) | PosPrune (LLaVA-1.5-7B) · MME and ScienceQA | score | Improved |
| PosPrune instead of VisionZip | LLaVA-1.5-7B · 8 image benchmarks (average) | performance retained: 96.7% -> 98.5% | Improved |
| PosPrune instead of VisionZip | Qwen2.5-VL-7B · 8 image benchmarks (average) | performance retained: 95.3% -> 97.8% | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- VisionZip — VisionZip: Longer is Better but Not Necessary in Vision Language Models (CVPR 2025) · better on 6 of 7 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Limitations
- Only the abstract was available; full method and results could not be verified.
Closest papers by meaning
- LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.95)
- QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.93)
- Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.93)
- TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model (CVPR 2026 · similarity 0.93)
- An Empirical Study of Attention and Diversity for Adaptive Visual Token Pruning in Large Vision-Language Models (ICLR 2026 · similarity 0.93)
- ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
