Research map / Vision-language and multimodal / Multimodal LLMs
FOLDER: Accelerating Multi-Modal Large Language Models with Enhanced Performance
Haicheng Wang, Zhemeng Yu, Gabriele Spadaro, Chen Ju, Victor Quétu, Shuai Xiao, Enzo Tartaglione
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| merge tokens only in the last vision block instead of uniformly in every block (ablation) | FOLDER on BLIP · BLIP captioning (60% reduction) | CIDEr: 128.8 -> 132.2 | Improved |
| merge tokens only in the last vision block instead of uniformly in every block (ablation) | FOLDER on BLIP · BLIP captioning (70% reduction) | CIDEr: 127.2 -> 131.0 | Improved |
| merge similar tokens by averaging instead of dropping them (ablation) | FOLDER on BLIP · BLIP captioning (50% reduction) | CIDEr: 130.0 -> 132.0 | Improved |
| use norm-weighted average instead of plain average (ablation) | FOLDER on BLIP · BLIP captioning (50% reduction) | CIDEr: 132.0 -> 132.2 | No clear effect |
| train LLaVA1.5-13B with FOLDER at 66% reduction | LLaVA1.5-13B · 9-benchmark average, inference at 66% | average score: 58.8 -> 61.2 | Improved |
| train LLaVA1.5-13B with FOLDER at 66% reduction | LLaVA1.5-13B · Hallusion Bench | accuracy: 36.1 -> 43.6 | Improved |
| raise training and inference reduction from 66% to 75% | LLaVA1.5-13B with FOLDER · 9-benchmark average | average score: 61.2 -> 58.2 | Hurt |
| use FOLDER instead of Turbo uniform merging at 66% reduction | LLaVA1.5-7B · OCRVQA | accuracy: 34.4 -> 46.1 | Improved |
| use FOLDER instead of FastV at 66% reduction (average over benchmarks) | LLaVA1.5-7B · 10-benchmark average | average score: 54.6 -> 54.8 | Mixed |
| use FOLDER instead of Turbo on Video-LLaVA at 65% reduction | Video-LLaVA · MME-Video | score: 29.8 -> 31.9 | Improved |
| add FOLDER with 66% reduction to LLaVA1.5-13B training | LLaVA1.5-13B · training time | GPU hours: 187 -> 124 | Improved |
| add FOLDER with 66% reduction before the LLM | LLaVA1.5-13B · inference peak memory | memory: 81.4G -> 48.7G | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Turbo — Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models (ECCV 2024) · better on 6 of 6 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings (NeurIPS 2025 · similarity 0.91)
- ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming (AAAI 2025 · similarity 0.90)
- Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference (AAAI 2025 · similarity 0.90)
- Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction (CVPR 2025 · similarity 0.90)
- Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models (AAAI 2025 · similarity 0.90)
- SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
