Research map / Vision-language and multimodal / Multimodal LLMs

Mostly Text, Smart Visuals: Asymmetric Text-Visual Pruning for Large Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ATV-Pruning, a Wanda-style post-training weight pruning method for LVLMs that builds an asymmetric calibration pool from all text tokens plus a block-adaptive subset of visual tokens selected by representation drift.

Problem

Activation-aware weight pruning of large vision-language models treats text and visual calibration tokens uniformly, ignoring that their pruning sensitivities differ, which degrades accuracy at high sparsity.

Approach

Decouple text and visual pathways in a MoT-style probe to show the textual pathway needs text-anchored calibration while the visual pathway is redundant; then calibrate with all text tokens plus a small layer-adaptive set of visual tokens chosen by cosine-based representation drift.

Key results

• At 50% unstructured sparsity on LLaVA-NeXT 8B, ATV-Pruning reaches 94.00% average retention vs 92.67% for TAMP and 88.36% for Wanda.
• At 60% sparsity on LLaVA-NeXT 8B, ATV-Pruning reaches 77.01% average retention, exceeding Wanda by +12.56 and TAMP by +3.26 points.
• At 60% sparsity on Qwen2-VL 7B, ATV-Pruning achieves 85.65% average retention, +0.35 over SparseGPT and +1.86 over TAMP.

Surprising findings

• Pruning only the visual pathway to 60% sparsity preserves over 99% of dense performance, even with text-only calibration.
• Text-only calibration at alpha=0 already outperforms Wanda's modality-agnostic pool on MMB (64.69 vs 63.83), and removing any text tokens hurts performance.

Limitations

• Evaluated mainly on 7-8B LVLMs with unstructured sparsity; semi-structured results are reported only for LLaVA-NeXT.
• Requires a hyperparameter alpha tuned per model (1.0 for LLaVA-NeXT, 1.5 for Qwen2-VL), and the adaptive selection adds overhead over Wanda.
• Sensitivity analysis is limited to three benchmarks and one backbone.

Experimental setup

Datasets: ShareGPT4V

Benchmarks: GQA MMBench-EN MME MMMU OK-VQA POPE ScienceQA-IMG TextVQA VizWiz

Metrics: official benchmark metric average performance retention

Baselines: SparseGPT Wanda TAMP

Builds on: Wanda SparseGPT TAMP Mixture-of-Transformers

Closest papers by meaning

• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.93)
• TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model (CVPR 2025 · similarity 0.92)
• ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning (AAAI 2026 · similarity 0.92)
• Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.92)
• ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.92)
• Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.92)