Research map / Vision-language and multimodal / Multimodal LLMs

Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FiCoCo, a training-free visual token reduction framework (FiCoCo-V in the vision encoder, FiCoCo-L in the LLM decoder) that filters redundant tokens, recycles their information into correlated kept tokens via self-preserving compression, and accelerates MLLM prefilling.

Problem

Quadratic prefilling cost of MLLMs with long multimodal contexts (many visual tokens) hinders deployment; existing training-free token reduction methods either prune information or use single, biased redundancy metrics and fixed merging counts.

Approach

Combine vision-aware (attention-received) and semantic-aware (CLS or mean-key) redundancy to filter tokens, use token-adaptive top-quantile correlation to decide which kept tokens receive each discarded token's information via dense pathways, and update kept tokens with a self-preserving weighted compression that keeps at least 50% of their original information; FiCoCo-L adds task-aware text-atten

Key results

• On LLaVA-NeXT-7B at 2.9 TFLOPs (93.2% reduction), FiCoCo-L retains 93.6% of vanilla average performance versus 89.4% for SparseVLM.
• On LLaVA-1.5-7B at 1.5 TFLOPs (82.4% reduction), both FiCoCo variants exceed 92% average retention, about 6% above the second-best SparseVLM.
• On Video-LLaVA at 2.6 TFLOPs (91.2% reduction), FiCoCo-V retains 92.8% versus 86.5% for SparseVLM and 52.1% for FastV.

Surprising findings

• Pruning with K=0 (no information recycling) outperforms fixed K=1 or K=2 merging in ablations, attributed to over-dilution from small fixed merge counts.
• Treating high inter-patch attention as redundancy (rather than importance) outperforms the attention-as-importance convention used by prior methods like FastV.

Limitations

• Token reduction starts only at a chosen layer (layer 12 in vision encoder, layer 4 in LLM), with hyperparameters tuned per method.
• FiCoCo-L's local penalty was found to hurt, and the method's advantage narrows under severe compression for some benchmarks.

Experimental setup

Datasets: SQA VQA-T POPE GQA MMB VQAv2 MMMU VizWiz MM-Vet MMB-CN LLaVA-W TGIF

Benchmarks: SQA VQA-T POPE GQA MMB VQAv2 MMMU VizWiz MM-Vet MMB-CN LLaVA-W TGIF

Metrics: accuracy average accuracy retention (%) TFLOPs throughput

Baselines: ToMe FastV SparseVLM PDrop PruMerge

Builds on: ToMe FastV SparseVLM

Closest papers by meaning

• VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration (ICLR 2026 · similarity 0.92)
• Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspective (ICLR 2026 · similarity 0.92)
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.91)
• Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings (NeurIPS 2025 · similarity 0.91)
• Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference (ICML 2026 · similarity 0.91)
• Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction (CVPR 2025 · similarity 0.90)