Research map / Vision-language and multimodal / Multimodal LLMs
Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models
Wei Suo, Ji Ma, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, Yanning Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DPO training to the layer-skipping router (ablation) | PAR layer pruning · POPE | accuracy: 81.4 -> 85.8 | Improved |
| add DPO training to the layer-skipping router (ablation) | PAR layer pruning · SEED-Bench-Image | accuracy: 61.6 -> 65.1 | Improved |
| add DPO training to token router (ablation) | PAR token pruning (T=176) · POPE | accuracy: 72.8 -> 74.9 | Improved |
| add Gaussian noise perturbation on top of DPO (ablation) | PAR token pruning (T=176) · POPE | accuracy: 74.9 -> 76.5 | Improved |
| add Gaussian noise perturbation on top of DPO (ablation) | PAR token pruning (T=176) · SEED-Bench-Image | accuracy: 61.2 -> 63.0 | Improved |
| add learned joint router (DPO and noise) instead of naive combination of layer skipping and token pruning (ablation) | PAR joint pruning · POPE | accuracy: 64.4 -> 76.1 | Improved |
| add learned joint router (DPO and noise) instead of naive combination of layer skipping and token pruning (ablation) | PAR joint pruning · SEED-Bench-Image | accuracy: 54.5 -> 62.7 | Improved |
| naively combine rule-based layer skipping with token pruning (ablation) | FastV plus layer skipping · POPE | accuracy: 72.8 -> 64.4 | Hurt |
| use PAR instead of FastV at same compute | LLaVA-1.5-7B · POPE | accuracy: 65.6 -> 75.9 | Improved |
| use PAR instead of ShortGPT-style layer skipping | LLaVA-1.5-7B · POPE | accuracy: 79.3 -> 82.9 | Improved |
| use specialized token-router instead of FastV | LLaVA-1.5-7B · SEED-Bench-Image | accuracy: 56.3 -> 59.0 | Improved |
| use one jointly trained router for token-only compression instead of retraining a token router | PAR · POPE | accuracy: 71.3 vs 72.7 | Mixed |
Closest papers by meaning
- ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.92)
- ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models (ICLR 2024 · similarity 0.92)
- Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.92)
- EfficientLLaVA: Generalizable Auto-Pruning for Large Vision-language Models (CVPR 2025 · similarity 0.92)
- SwiftVLM: Efficient Vision-Language Model Inference via Cross-Layer Token Bypass (NeurIPS 2026 · similarity 0.92)
- OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization (ICML 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
