Research map / Vision-language and multimodal / Multimodal LLMs

VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VisionTrim, a training-free, plug-and-play visual token compression framework that prunes/merges tokens in both the vision encoder and LLM decoder of MLLMs using global-local token scoring (DVTS) and text-guided merging (TGVC).

Problem

MLLMs incur high inference cost from excessive visual tokens, especially for high-resolution images and video; existing token reduction methods act on only one pipeline stage and ignore text-visual alignment, degrading accuracy.

Approach

Combines [CLS]-attention global importance with a dual-kernel local affinity (feature and spatial proximity) using adaptive variance-based weighting to select dominant tokens, then uses text-token similarity to pick cluster centers and merge the discarded tokens into complement tokens; applied at both the vision encoder and LLM layers.

Key results

• On LLaVA-1.5-7B with 64 tokens (88.9% reduction), VisionTrim retains 98.8% of vanilla average performance vs. 96.8% for VScan and 94.4% for VisionZip.
• On LLaVA-NeXT-7B retaining 22.2% of tokens (640), VisionTrim achieves 99.9% of original performance; at 320 tokens it reaches 97.0% vs. 94.1% for VisionZip.
• At 88.9% reduction on LLaVA-NeXT-7B, VisionTrim reduces CUDA time by 61.4%, FLOPs by 91.7%, and KV cache by 93.3%, with 2.48x total inference speedup on POPE.

Surprising findings

• With only 1 retained visual token (99.8% reduction), VisionTrim still keeps about 82.8% of original average performance without training.
• On Qwen2-VL-7B, the training-free method exceeds the vanilla model on MMBench by 2.1% at about one-third of the tokens.

Limitations

• Authors acknowledge the method is not lossless; a performance drop remains at high reduction ratios (e.g., 98.8% at 88.9% reduction).
• Video results use a fixed 136-token budget following SparseVLM; gains on video are smaller relative to image settings.
• Fine-tuned variant requires training of TGVC attention module and projector, outside the training-free setting.

Experimental setup

Datasets: GQA VQAv2 VizWiz POPE MMBench MME MM-Vet TextVQA SEED ScienceQA TGIF-QA MSVD-QA

Benchmarks: GQA MMBench MME POPE ScienceQA VQAv2 TextVQA SEED MM-Vet VizWiz TGIF-QA MSVD-QA

Metrics: accuracy relative performance (% of vanilla) CUDA time FLOPs KV cache memory prefill time BLEU@4 CIDEr throughput

Baselines: SparseVLM VisionZip PyramidDrop VScan FastV Turbo TopV

Builds on: FasterVLM VisionZip FastV SparseVLM ToMe

Closest papers by meaning

• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.93)
• Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration (AAAI 2026 · similarity 0.92)
• DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference (CVPR 2026 · similarity 0.92)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.91)
• Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspective (ICLR 2026 · similarity 0.91)
• Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.91)