Research map / Vision-language and multimodal / Multimodal LLMs
FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlowCut, a training-free visual token pruning framework for LVLMs that uses information-flow analysis (CLS token as relay, attention-entropy-adaptive prune ratios, multi-criteria scores, cumulative cross-layer importance) to prune vision tokens.
Problem
Existing visual token pruning methods for LVLMs rank and prune tokens using attention scores from a single layer, which may fail to identify redundancy because token-layer interactions are complex.
Approach
Models redundancy as an emerging property of information flow: the CLS token acts as a global relay, attention entropy sets layer-wise prune ratios, and importance combines attention strength, CLS-value semantic similarity, and value L1 norm, accumulated across layers with a history-current moving average.
Key results
- On LLaVA-1.5-7B with 64 retained tokens (88.9% reduction), FlowCut reaches 96.0% of upper-bound average vs 94.4% for VisionZip.
- On LLaVA-NeXT-7B with 160 retained tokens (94.4% reduction), FlowCut reaches 91.9% vs 87.6% for VisionZip, with 3.2x prefilling speedup.
- On Video-LLaVA with 256 total tokens, FlowCut reaches 98.6% average vs 94.4% for VisionZip.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FlowCut instead of VisionZip at 64 retained tokens | LLaVA-1.5-7B · Image understanding benchmarks | average accuracy proportion: 94.4% -> 96.0% | Helped |
| Use FlowCut instead of VisionZip at 160 retained tokens | LLaVA-NeXT-7B · Image understanding benchmarks | average accuracy proportion: 87.6% -> 91.9% | Helped |
| Use FlowCut instead of VisionZip | Video-LLaVA · TGIF-QA, MSVD-QA, MSRVTT-QA | average accuracy proportion: 94.4% -> 98.6% | Helped |
| Prune to 160 visual tokens with FlowCut instead of using the unpruned model | LLaVA-NeXT-7B · POPE benchmark | prefilling speedup: 3.2× | Helped |
| Train with 192 visual tokens using FlowCut instead of 576 | LLaVA-1.5-7B · 10 image understanding benchmarks | average accuracy proportion: 100% -> 101.1% | Helped |
| Use cumulative historical and current importance scores instead of only the current score (ablation) | FlowCut · LLaVA-1.5-7B POPE | POPE: 83.8 -> 85.2 | Helped |
| Prune every 2 layers instead of pruning only at the last layer (ablation) | FlowCut · LLaVA-1.5-7B POPE | POPE: 83.9 -> 85.2 | Helped |
| Prune every layer instead of every 2 layers (ablation) | FlowCut · LLaVA-1.5-7B POPE | POPE: 85.2 -> 84.9 | Hurt |
| Increase the pruning interval from 2 to 3 layers (ablation) | FlowCut · LLaVA-1.5-7B POPE | POPE: 85.2 -> 84.6 | Hurt |
| Use the 0.6:0.4 history-to-current score weighting instead of 0.5:0.5 (ablation) | FlowCut · LLaVA-1.5-7B POPE | POPE: 85.2 -> 85.2 | No clear effect |
| Add cumulative importance evaluation (ablation) | FlowCut · Three image understanding benchmarks, 128 retained tokens | benchmark performance | Helped |
| Add multi-criteria scoring (ablation) | FlowCut · Three image understanding benchmarks, 128 retained tokens | benchmark performance | Helped |
Surprising findings
- Pruning sometimes yields more accurate answers than the unpruned baseline, which the authors attribute to removing distracting tokens.
- Applying the method during training with 192 tokens reportedly improves LLaVA-1.5 performance to 101.1% of the original model while cutting training time by 1.5x.
Limitations
- Relies on the CLS token (or a mean-pooled global token) as a proxy for overall information flow, which is an empirical approximation.
- Multi-criteria weighting and 0.5/0.5 cumulative coefficients are hand-set hyperparameters, selected with limited sensitivity analysis.
Experimental setup
Datasets: GQA MMBench MME POPE ScienceQA VQA-V2 TextVQA VizWiz SEED-Bench MM-Vet LLaVA-Bench TGIF-QA
Benchmarks: GQA MMBench MMBench-CN MME POPE ScienceQA VQA-V2 TextVQA VizWiz SEED-Bench MM-Vet LLaVA-Bench
Metrics: accuracy relative average performance (% of upper bound) score TFLOPs prefilling time total inference time
Baselines: FastV SparseVLM PDrop VisionZip ToMe ST^3 VTW RCom FasterVLM LLaVA-PruMerge
Builds on: FastV VisionZip PDrop SparseVLM
Closest papers by meaning
- Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.92)
- Mostly Text, Smart Visuals: Asymmetric Text-Visual Pruning for Large Vision-Language Models (CVPR 2026 · similarity 0.91)
- Each Complexity Deserves a Pruning Policy (NeurIPS 2025 · similarity 0.91)
- ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.91)
- LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.91)
- An Empirical Study of Attention and Diversity for Adaptive Visual Token Pruning in Large Vision-Language Models (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.