Research map / Vision-language and multimodal / Multimodal LLMs

D²Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruning

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes D²Pruner, a training-free visual token pruning method for MLLMs that combines a positionally debiased attention importance score with a graph-based maximal independent set selection over a hybrid spatial-semantic token graph.

Problem

Visual token pruning for MLLMs works on general understanding but fails on fine-grained localization; importance-based methods are distorted by positional bias in attention, and diversity-based methods ignore 2D spatial structure and the prompt.

Approach

Divides attention by an averaged positional-bias prior (computed over 1,000 COCO images) to get a relative importance score, then selects pivot tokens by that score and fills the rest via greedy MIS selection on a graph whose edges combine spatial 8-neighborhood adjacency and cosine semantic similarity.

Key results

• On LLaVA-1.5-7B with 192 tokens (66.7% reduction), D²Pruner retains 99.9% of average performance; at 64 tokens it retains 97.3%.
• On InternVL-2.5-8B at 90% token reduction, RefCOCO-family localization average is 85.68% versus 22.1%-40.65% for competing methods.
• On LLaVA-Next-7B with 11.2% visual tokens kept, prefill speeds up 5.09x and KV cache shrinks 7.63x, with pruning overhead of 0.8% of a forward pass.

Surprising findings

• Averaged attention maps show content-agnostic positional bias that differs by model: bottom-focused for LLaVA (absolute embeddings) and periphery-focused for InternVL and Qwen-VL (relative embeddings).
• Diversity-only method DivPrune collapses on RefCOCO localization for LLaVA-1.5-7B at 50% or higher reduction, while FastV is competitive at moderate ratios.

Limitations

• Relies on a one-time positional bias prior calibrated on 1,000 COCO images, which may not transfer to other domains or resolutions without re-calibration.
• Hyperparameters (alpha, lambda, theta_sim, r_pivot, pruning layer K) are set per task and model, and sensitivity is only partly explored.

Experimental setup

Datasets: COCO

Benchmarks: GQA MMBench MMBench-CN MME POPE ScienceQA VQAv2 TextVQA VizWiz AI2D ChartQA DocVQA

Metrics: accuracy relative performance retention (%) FLOPs prefill time KV cache size

Baselines: FastV DART DivPrune ToMe HiRED FitPrune LLaVA-PruMerge SparseVLM PDrop VisionZip FasterVLM GlobalCom$^2$

Builds on: FastV DART DivPrune RoPE

Closest papers by meaning

• Don't Just Chase “Highlighted Tokens” in MLLMs: Revisiting Visual Holistic Context Retention (NeurIPS 2025 · similarity 0.91)
• Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs (NeurIPS 2025 · similarity 0.91)
• What Kind of Visual Tokens Do We Need? Training-Free Visual Token Pruning for Multi-Modal Large Language Models from the Perspective of Graph (AAAI 2025 · similarity 0.91)
• TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model (CVPR 2026 · similarity 0.90)
• QuietPrune: Query-Guided Early Token Pruning for Vision-Language Models (CVPR 2026 · similarity 0.90)
• Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models (AAAI 2025 · similarity 0.90)