Research map / Vision-language and multimodal / Multimodal LLMs

Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GlobalCom2, a training-free token compression method for dynamic-cropping high-resolution LVLMs that uses the global thumbnail to guide per-crop compression ratios and token selection.

Problem

Token compression methods designed for single-view LVLMs ignore the multi-view structure of high-resolution LVLMs with dynamic cropping, leading to over-compression of informative crops and positional bias under aggressive compression.

Approach

Use the global thumbnail's [CLS]-to-patch attention as a holistic guide: allocate per-crop retention ratios via softmax over thumbnail-region importance sums, and score tokens by blending thumbnail-derived global importance with local [CLS] attention. A [CLS]-free variant uses cosine dissimilarity to the mean token.

Key results

• On LLaVA-NeXT-7B at 10% retention (288 tokens), GlobalCom2 reaches 91.6% average performance, versus 85.4% for FastV and 89.9% for FasterVLM.
• Plugging GlobalCom2 into FastV and SparseVLM at 10% retention yields average gains of 5.3% and 5.2%, with POPE gains of 8.2 and 4.5.
• On LLaVA-NeXT at 10% retention, GlobalCom2 cuts TFLOPs from 41.7 to 3.8 (90.9% reduction) and peak memory from 23.0 to 13.9 GB, with 1.8x throughput over the vanilla model.

Surprising findings

• Using only the global thumbnail yields better general perception performance than the full dynamic-cropping input on some benchmarks, while local crops excel at fine-grained tasks.
• FastV and PDrop show severe POPE degradation under extreme compression, attributed to positional bias toward later crops.

Limitations

• Hyperparameters such as alpha=0.5 and tau=10 were set empirically.
• Crop-level scoring for models without [CLS] relies on an alternative cosine-dissimilarity measure whose analysis is deferred to the appendix.
• On LLaVA-NeXT at 25% retention, GlobalCom2 does not win on every benchmark (e.g., MMBench and VizWiz trail some baselines).

Experimental setup

Datasets: SQA MMBench TextVQA POPE GQA VizWiz MME MM-Vet SEED

Benchmarks: GQA VizWiz SQA MMBench POPE TextVQA MME MM-Vet SEED

Metrics: accuracy relative performance (% of upper bound) TFLOPs peak GPU memory throughput

Baselines: FastV SparseVLM PDrop PruMerge FasterVLM

Builds on: FastV SparseVLM FasterVLM LLaVA-NeXT

Closest papers by meaning

• LLMC+: Benchmarking Vision-Language Model Compression with a plug-and-play Toolkit (AAAI 2026 · similarity 0.90)
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.89)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.89)
• iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models (ICLR 2026 · similarity 0.89)
• Hybrid Token Compression for Vision-Language Models (CVPR 2026 · similarity 0.89)
• From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection (ICML 2025 · similarity 0.88)