Research map / Vision-language and multimodal / Multimodal LLMs

LLMC+: Benchmarking Vision-Language Model Compression with a plug-and-play Toolkit

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces LLMC+, a plug-and-play toolkit and benchmark for training-free VLM compression that compares token reduction (spatial and temporal) and quantization, with new practical evaluations on fine-grained and multi-turn tasks.

Problem

Training-free VLM compression lacks fair, modular comparison across spatial and temporal token reduction, is evaluated only on simple single-turn VQA, and rarely studies joint token reduction plus model-level compression.

Approach

A modular toolkit that decomposes token reduction into metric (attention vs. similarity), solution (prune vs. merge), and video segmentation (fixed vs. dynamic) components, benchmarked under a unified setup together with fine-grained (DocVQA, OCRBench) and multi-turn consistency evaluations and joint quantization plus token reduction.

Key results

• On LLaVA-1.5-7B at 192 tokens, VisionZip PA retains 97.9% relative accuracy while ToMe MS retains 95.8%, supporting prune-over-merge in the vision tower.
• On DocVQA and OCRBench with LLaVA-NeXT-7B at ~160 tokens, accuracy falls to roughly 50% range for several methods (e.g., VisionZip DOC 28.8 vs. vanilla 68.5 at 160 tokens on DocVQA, and VisionZip OCR 29.5 vs. 52.1).
• Combining FastV with SmoothQuant W8A8 on LLaVA-1.5-7B yields 92.9% average versus 93.4% for FastV alone, and on LLaVA-NeXT the combined setup reaches 2.54x speedup and 1.64x memory reduction.

Surprising findings

• Attention-based text-guided token reduction in the LLM (FastV, SparseVLM) performs worse than text-agnostic vision-tower methods in multi-turn dialogue, as they prune tokens needed in later turns.
• Temporal merge slightly outperforms temporal prune, contrary to the common assumption that dropping redundant frame tokens is sufficient.

Limitations

• Multi-turn evaluation relies on a GQA-derived two-turn construction rather than a naturally occurring dialogue dataset.
• Model-level compression study is limited to GPTQ and SmoothQuant and a few token reduction methods on LLaVA-1.5.

Experimental setup

Datasets: GQA GQA-derived two-turn dialogue pairs MMBench MME POPE TextVQA VizWiz ScienceQA OCRBench DocVQA MVBench LongVideoBench

Benchmarks: GQA ScienceQA TextVQA VizWiz MMBench MME POPE OCRBench DocVQA MVBench LongVideoBench MLVU

Metrics: accuracy relative accuracy prefill latency peak memory conditional multi-turn consistency P(Q2|Q1)

Baselines: FastV SparseVLM VisionZip VisPruner DivPrune ToMe MustDrop DART HoliTom GPTQ SmoothQuant

Builds on: FastV SparseVLM VisionZip DART LMMs-Eval

Closest papers by meaning

• VoCo-LLaMA: Towards Vision Compression with Large Language Models (CVPR 2025 · similarity 0.92)
• VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration (ICLR 2025 · similarity 0.91)
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.91)
• Hybrid Token Compression for Vision-Language Models (CVPR 2026 · similarity 0.91)
• Are We on the Right Way for Evaluating Large Vision-Language Models? (NeurIPS 2024 · similarity 0.91)
• DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference (CVPR 2026 · similarity 0.91)