Research map / Vision-language and multimodal / Multimodal LLMs

CoIn: Coverage and Informativeness-Guided Token Reduction for Efficient Large Multimodal Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CoIn, a training-free visual token reduction method for LMMs that selects a token subset by jointly maximizing informativeness (visual saliency plus text alignment) and volume-based coverage via greedy submodular selection.

Problem

Visual tokens in large multimodal models create a large inference and memory bottleneck; existing importance-based and diversity-based token pruning methods each have drawbacks (attention bias, incompatibility with FlashAttention, encoder dependence, redundancy, ignoring saliency).

Approach

Reformulates visual token reduction as a subset selection problem that jointly optimizes per-token informativeness (feature-norm saliency plus cosine alignment with mean text embedding) and a log-determinant volume coverage term, solved greedily in a single stage.

Key results

• On LLaVA-1.5-13B with 32 retained tokens (94.4% reduction), CoIn retains 91.0% of full-model average performance across 9 benchmarks.
• On LLaVA-NeXT-7B with 160 tokens, CoIn retains 86.7% average, outperforming VisionZip by 3.2%.
• On LLaVA-NeXT-7B POPE efficiency test, CoIn achieves 6.5x prefill speedup (233ms to 36ms) and 14.1GB vs 16.8GB peak GPU memory.

Surprising findings

• Informativeness-only and coverage-only selection both perform much worse than their combination (e.g., POPE 73.7 and 74.4 vs 86.2 combined), and CoIn surpasses the unpruned model on POPE at 64 tokens.

Limitations

• Hyperparameters alpha and beta must be tuned per model and token budget; best settings vary across tasks.
• Video experiments use a fixed-size benchmark setup and DyCoke comparison is only on its token reduction component.

Experimental setup

Benchmarks: GQA MMBench MME POPE TextVQA VizWiz OCRBench ScienceQA-IMG RealWorldQA AI2D MVBench VSI-Bench

Metrics: accuracy relative performance average GPU memory prefill time decoding time

Baselines: PDrop SparseVLM DART DivPrune CDPruner VisionZip PruMerge FrameFusion DyCoke

Builds on: volume-based subset selection FlashAttention DivPrune VisionZip

Closest papers by meaning

• DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models (CVPR 2025 · similarity 0.90)
• MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs (ICLR 2026 · similarity 0.90)
• SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs (NeurIPS 2025 · similarity 0.90)
• One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs (CVPR 2026 · similarity 0.89)
• TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model (CVPR 2026 · similarity 0.89)
• Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.89)