Research map / Vision-language and multimodal / Multimodal LLMs
SPLIT-VLM: Salience-Guided Partitioning towards Local Coverage for Importance-Aware Token Dropping in Vision-Language Models
Seungil Lee, Gilha lee, Hyun Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPLIT-VLM, a salience-guided partitioning method for importance-aware visual token dropping in vision-language models to reduce compute and memory.
Problem
Vision tokens dominate compute and memory in VLMs; prior token-reduction methods trade off salience (position bias, extra compute) against diversity (under-covers salient regions, hyperparameter-sensitive).
Approach
Combines salience and local coverage via salience-guided partitioning to choose which vision tokens to drop, aiming to get benefits of both salience and diversity without their stated drawbacks. Full details unavailable beyond the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Temporal Shift for allocation and Diversity Score for selection instead of Diversity Score for both (ablation) | SPLIT · Qwen2-VL-7B, 88.9% reduction | Average accuracy (%): 85.0 -> 85.6 | Improved |
| use Temporal Shift for allocation instead of Diversity Score (selection by Diversity Score) (ablation) | SPLIT · Qwen2-VL-7B, 88.9% reduction | Average accuracy (%): 85.0 -> 85.6 | Improved |
| use Diversity Score for selection instead of Temporal Shift (allocation by Temporal Shift) (ablation) | SPLIT · Qwen2-VL-7B, 88.9% reduction | Average accuracy (%): 85.2 -> 85.6 | Improved |
| add region-wise budget allocation to VisPruner (ablation) | VisPruner · LLaVA-1.5-7B, 64 tokens | Average relative accuracy (%): 91.4 -> 91.7 | Improved |
| add region-wise budget allocation to VisionZip (ablation) | VisionZip · LLaVA-1.5-7B, 64 tokens | Average relative accuracy (%): 90.5 -> 91.0 | Improved |
| add region-wise budget allocation to CDPruner (ablation) | CDPruner · LLaVA-1.5-7B, 64 tokens | Average relative accuracy (%): 92.4 -> 93.0 | Improved |
| add region-wise budget allocation to CDPruner (imbalance Gini) (ablation) | CDPruner · LLaVA-1.5-7B, 64 tokens | Gini coefficient of retained tokens: 0.325 -> 0.269 | Improved |
| use importance-based adaptive allocation instead of uniform or random allocation (ablation) | region budget · LLaVA-1.5-7B (POPE, MME) | POPE / MME | Improved |
| use 16 regions instead of finer partitioning (e.g., 6x6) at 64 tokens (ablation) | SPLIT · LLaVA-1.5-7B, 64 tokens | accuracy | Improved |
| compute temporal shift across all ViT layers instead of a subset (ablation) | SPLIT · LLaVA-1.5-7B | accuracy | Improved |
| add cross-modal importance to Diversity Score selection after an early LLM layer (SPLIT-LLM B) (ablation) | SPLIT · Qwen2-VL-7B, 88.9% reduction | Average accuracy (%): 85.6 -> 86.0 | Improved |
| use cross-modal similarity for region budget allocation (SPLIT-LLM A) (ablation) | SPLIT · Qwen2-VL-7B, 88.9% reduction | Average accuracy (%): 85.6 -> 85.6 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- DivPrune — DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models (CVPR 2025) · better on 9 of 9 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SCoRe: Salience-Coverage Reduction for Vision Token Pruning in Vision-Language Models (CVPR 2026 · similarity 0.91)
- Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.91)
- MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs (ICLR 2026 · similarity 0.91)
- Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.91)
- TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model (CVPR 2026 · similarity 0.90)
- SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
