Research map / Vision-language and multimodal / Multimodal LLMs
Training-free token reduction for MLLMs
Searches or selects which vision tokens to trim or merge across layers and frames without retraining.
90 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2024–2026) · an approach in Multimodal LLMs, keyword cluster “tokens · pruning · visual token”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 4 of 4 tracked benchmarks.
7 papers in 2023–24 and 83 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.4% to 3.8%.
Open questions the data points to
- GQA (accuracy): best went from 56.8 (up to 2024) to 65.2 in 2025 — Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models (ICLR 2025 · 3 citations)
- MME (score): best went from 1311 (up to 2024) to 1872 in 2025 — Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference (AAAI 2025 · 26 citations)
- MMBench (score): best went from 63.4 (up to 2024) to 69.5 in 2025 — Matryoshka Multimodal Models (ICLR 2025 · 0 citations)
- MMBench (accuracy): best went from 72.8 (up to 2024) to 75.7 in 2025 — FlashSloth : Lightning Multimodal Large Language Models via Embedded Visual Compression (CVPR 2025 · 3 citations)
- Papers report LLaVA-1.5-7B on MME (score) anywhere from 1496 to 1866: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2024 · Efficient Large Multi-modal Models via Visual Context Compression (NeurIPS 2024 · 8 citations)
- 2025 · DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models (CVPR 2025 · 27 citations)
- 2026 · Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration (AAAI 2026 · 2 citations)
Earliest papers here
- Efficient Large Multi-modal Models via Visual Context Compression (NeurIPS 2024 · 8 citations)
- ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models (NeurIPS 2024 · 5 citations)
- Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs (NeurIPS 2024 · 4 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GQA · accuracy (higher is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 56.8 | Auto-Encoding Morph-Tokens for Multimodal LLM (ICML 2024 · 0 citations) |
| 2025 | 65.2 | Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models (ICLR 2025 · 3 citations) |
MME · score (higher is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 1311 | Visual Perception by Large Language Model’s Weights (NeurIPS 2024 · 1 citation) |
| 2025 | 1872 | Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference (AAAI 2025 · 26 citations) |
MMBench · score (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 63.4 | Visual Perception by Large Language Model’s Weights (NeurIPS 2024 · 1 citation) |
| 2025 | 69.5 | Matryoshka Multimodal Models (ICLR 2025 · 0 citations) |
MMBench · accuracy (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 72.8 | Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight (NeurIPS 2024 · 1 citation) |
| 2025 | 75.7 | FlashSloth : Lightning Multimodal Large Language Models via Embedded Visual Compression (CVPR 2025 · 3 citations) |
What has been tried
37 papers here report 442 measured changes (a component added, removed or swapped, compared with a baseline): helped 61%, no clear effect 10%, hurt 26%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| prune after decoder layer 1 rather than before the first decoder layer | Average across MMB, MMMU, POPE, SQA · average score | 62.34 -> 60.44 | Hurt | DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models (CVPR 2025 · 27 citations) |
| random-layer withdrawal (K random in [8,24]) instead of selected K | MME · score | -124.76 vs VTW K=16 (1872.43 to 1741.34) | Hurt | Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference (AAAI 2025 · 26 citations) |
| increase FitPrune pruning ratio from 0% to 50% | GQA · accuracy | 62.0 -> 61.9 | No clear effect | Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models (AAAI 2025 · 23 citations) |
| Ignore 124/576 image tokens | ScienceQA (Image) · accuracy | 65.15 -> 68.02 | Helped | Enhancing Multimodal Large Language Models Complex Reason via Similarity Computation (AAAI 2025 · 17 citations) |
| add similarity points and DtoC to the <SEG> prompt | ReasonSeg val · cIoU | 57.6 -> 67.6 | Helped | Reasoning to Attend: Try to Understand How <SEG> Token Works (CVPR 2025 · 10 citations) |
| add APS to LLaVA1.5 | DocVQA val · accuracy | 8.5 -> 10.7 (+27.4%) | Helped | DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming (AAAI 2025 · 9 citations) |
| training-free average pooling of visual tokens at test time (stride 4 from layer 16, ~70% tokens removed) | GQA · GQA accuracy | ~3% drop | Hurt | Efficient Large Multi-modal Models via Visual Context Compression (NeurIPS 2024 · 8 citations) |
| use T=3 optimization steps rather than T=0 | ROC validation set, box · accuracy | 57.00 -> 61.00 | Helped | ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models (NeurIPS 2024 · 5 citations) |
| Freeze the LLM rather than train it during multimodal pretraining | Multimodal input alignment · cross-modal similarity | Hurt | Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs (NeurIPS 2024 · 4 citations) | |
| apply text-guided visual token recovery instead of visual-score-only pruning | ScienceQA · score | 68.72 -> 68.91 | Helped | Recoverable Compression: A Multimodal Vision Token Recovery Mechanism Guided by Text Information (AAAI 2025 · 3 citations) |
| Use cosine VTA decay with τ=50 instead of exponential decay with σ=20 | COCO2017 caption validation · CIDEr | 110.29 -> 110.66 | Helped | ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming (AAAI 2025 · 3 citations) |
| Add L2-norm token scoring to graph propagation | TextVQA · accuracy | 64.01 -> 64.05 | No clear effect | What Kind of Visual Tokens Do We Need? Training-Free Visual Token Pruning for Multi-Modal Large Language Models from the Perspective of Graph (AAAI 2025 · 3 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| LLaVA-1.5-7B | MME | score | 1496 – 1866 | 2 papers, e.g. ICLR 2025 and AAAI 2025 |
What reviewers will expect
Datasets and baselines named in the 83 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- POPE · 18 of 83 papers
- MME · 17 of 83 papers
- TextVQA · 17 of 83 papers
- MMBench · 16 of 83 papers
- GQA · 16 of 83 papers
- ScienceQA · 11 of 83 papers
- VQAv2 · 10 of 83 papers
- VizWiz · 9 of 83 papers
Baselines
- FastV · 23 of 83 papers
- SparseVLM · 14 of 83 papers
- VisionZip · 14 of 83 papers
- ToMe · 10 of 83 papers
- DART · 9 of 83 papers
- PruMerge · 8 of 83 papers
- InstructBLIP · 7 of 83 papers
- PDrop · 7 of 83 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 90 papers; from OpenAlex and the papers' affiliations).
- Xiamen University · 7 papers
- Shanghai Jiao Tong University · 7 papers
- Zhejiang University · 7 papers
- Huawei · 4 papers
- Peking University · 4 papers
- Tsinghua University · 4 papers
- Chinese Academy of Sciences · 3 papers
- Tencent · 3 papers
- University of Science and Technology of China · 3 papers
- National University of Singapore · 3 papers
Use it
Download BibTeX of 20 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Attention-based visual token pruning (117 papers)
Elsewhere in the topic
- Chain-of-thought multimodal reasoning (332 papers)
- VLM capability analysis (264 papers)
- Multimodal LLM benchmarks (252 papers)
- Spatial reasoning in VLMs (207 papers)
- Instruction-tuned multimodal assistants (199 papers)
- Benchmarks and analysis of MLLMs (181 papers)
- Reinforcement learning for visual reasoning (163 papers)
- Interleaved image-text generation with MLLMs (153 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).