Research map / Vision-language and multimodal / Multimodal LLMs

Collaborative Multi-Mode Pruning for Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CoMP, a joint parameter-and-token pruning framework for VLMs that couples importance metrics (CIM) and adaptively schedules pruning modes via cost-aware, exploratory strategy (MPS), outperforming single-mode and simple joint baselines at high pruning ratios.

Problem

Single-mode pruning (only parameters or only tokens) of Vision-Language Models leads to substantial performance degradation at high pruning ratios, and naive joint pruning suffers from inconsistent parameter/token importance and rigid pruning-mode ordering.

Approach

Jointly estimates parameter importance weighted by token importance and token importance corrected by the parameter mask (CIM), and selects the next pruning mode among five modes (vision/language/cross-modal parameters, vision/language tokens) by an EMA of validation-accuracy change per FLOP, with random exploration and interval-weighted historical information (MPS).

Key results

• On NLVR2 at pruning ratio 0.85, CoMP reaches 76.08% test accuracy vs 72.57% for MADTP (+3.51%).
• On LLaVA-v1.5-7B at 0.62 pruning, CoMP (with SFT) averages 66.98 vs 65.92 for DART†.
• On BLIP COCO retrieval at 0.7 pruning, CoMP improves I→T R@1 to 76.2 vs 73.9 for MADTP.

Surprising findings

• Naive simple joint pruning of parameters and tokens only matches single-mode pruning performance.

Limitations

• Designed for medium-scale task-specific VLMs where full-parameter training is tolerable; LLM-based extension is limited to one SFT epoch.
• Multiple hyperparameters (rho, tau, lambda0, I_max, interval I) introduced by MPS.

Experimental setup

Datasets: NLVR2 COCO Flickr30K VQAv2 Mix665K

Benchmarks: NLVR2 COCO Flickr30K VQAv2 MME MMBench GQA TextVQA POPE MSR-VTT

Metrics: accuracy R@1 R@5 CIDEr SPICE GFLOPs

Baselines: UPop MADTP STP Turbo M-Pruning FastV SparseVLM PDrop VisionZip DART DivPrune

Builds on: UPop MADTP Wanda PDrop BLIP

Closest papers by meaning

• ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning (AAAI 2026 · similarity 0.91)
• MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer (CVPR 2024 · similarity 0.91)
• Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization (NeurIPS 2025 · similarity 0.91)
• ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models (CVPR 2025 · similarity 0.91)
• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.91)
• ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models (ICLR 2024 · similarity 0.90)