Research map / Vision-language and multimodal / Multimodal LLMs

PPE: Positional Preservation Embedding for Token Compression in Multimodal Large Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PPE, a parameter-free positional encoding operator that lets each compressed visual token retain multiple original RoPE/M-RoPE position IDs, enabling spatiotemporal-preserving token merging and cascade compression in MLLMs.

Problem

Visual token merging in MLLMs shortens sequences but disrupts spatial layouts and temporal continuity because each merged token keeps only one (or a randomized/center) position ID, hurting layout-sensitive and video tasks.

Approach

Split the RoPE/M-RoPE embedding dimensions into K chunks so each merged token carries up to K original position IDs (top-K by distance to cluster center), with the same rotation applied as standard RoPE; combine this with cascade clustering inside LLM layers.

Key results

• On Qwen2.5-VL-3B with 55% spatial token reduction, PPE averages 79.32 on image benchmarks vs 67.61 for Chat-UniVi (dense 82.96), and 58.69 vs 56.74 on video tasks.
• With 90% reduction via within-LLM cascade (PPE*), VideoMME (w/o subs) reaches 58.48 vs 56.41 for the cascade baseline without PPE.
• Integrated into training-free PACT, PPE improves MMBench-EN from 74.14 to 74.48 and TextVQA from 73.73 to 73.87 with no added memory.

Surprising findings

• Chat-UniVi without SFT performs very poorly on layout-sensitive benchmarks (e.g., TextVQA 37.60), while adding PPE recovers it to 73.78.
• On the 7B model, VisionZip+PPE slightly decreases ChartQA in the training-free setting (-0.68).

Limitations

• PPE underperforms the dense model on some benchmarks and under aggressive 90% compression, with failure cases on fine-grained OCR text.
• Gains are reported mainly on a 3B Qwen2.5-VL with down-sampled SFT data; the 7B experiments used only 1/5 of the data.
• Choice of K is tied to the GCD of M-RoPE sections, with limited theoretical grounding shown beyond heuristic analysis.

Experimental setup

Datasets: LLaVA-Video-178k LLaVA-OneVision Flickr30k

Benchmarks: MMBench SQA TextVQA DocVQA OCRBench ChartQA VideoMME NeXT-QA SEED-Bench-Video MVBench Flickr30k

Metrics: accuracy CIDEr BLEU METEOR ROUGE_L SPICE latency (s) peak memory (GB)

Baselines: Chat-UniVi PACT SparseVLM ToMe VisionZip

Builds on: Chat-UniVi RoPE M-RoPE PACT VisionZip

Closest papers by meaning

• CORE: Compact Object-centric REpresentations as a New Paradigm for Token Merging in LVLMs (CVPR 2026 · similarity 0.91)
• EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling (CVPR 2026 · similarity 0.91)
• Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspective (ICLR 2026 · similarity 0.91)
• Hybrid Token Compression for Vision-Language Models (CVPR 2026 · similarity 0.90)
• VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration (ICLR 2026 · similarity 0.90)
• What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Models (CVPR 2026 · similarity 0.90)