Research map / Vision-language and multimodal / Multimodal LLMs

G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes G²TR, a training-free, plug-and-play visual token reduction method for separate-encoder unified multimodal models that scores understanding-side ViT tokens by cosine consistency with VAE latent anchors, then balanced-selects and merges removed tokens.

Problem

Visual token reduction for separate-encoder UMMs is unexplored; existing text-centric pruning (attention or text-image similarity) ignores that understanding-side tokens also condition image editing, causing editing quality loss.

Approach

Use the generation branch's VAE latents as the guidance signal: score each ViT token by cosine similarity to its spatially matched VAE latent anchor, select tokens with balanced per-anchor selection, and merge removed tokens into their nearest retained tokens before LLM prefill.

Key results

• At 50% visual tokens on BAGEL-7B, G²TR reaches 99.0% understanding relative average (vs 98.6% for IVC-Prune) and 1.94x prefill FLOPs reduction.
• On InternVL-U at 50% tokens, G²TR achieves 94.9% relative average, above IVC-Prune at 93.7%.
• On image editing, G²TR reaches 98.0% relative average on BAGEL-7B, versus 94.0% for IVC-Prune and 96.5% for FastV.

Surprising findings

• Different text-centric token reduction rules yield only a narrow spread on understanding benchmarks (e.g., MMBench 85.7 to 88.5), suggesting complex text-centric methods may not be cost-effective for UMMs.
• Removing high-score tokens provides no consistent gain over random removal, while removing low-score tokens does.

Limitations

• Fixed token budget; no dynamic reduction ratio based on task or latency.
• Only evaluated on two separate-encoder UMMs (BAGEL-7B-MoT and InternVL-U); may not extend to unified-encoder designs.
• No error bars or statistical significance reported; results from single runs.

Experimental setup

Datasets: MME MMBench MMVP RealWorldQA GEdit-Bench IntelligentBench RISE-Bench

Benchmarks: MME MMBench MMVP RealWorldQA GEdit-Bench IntelligentBench RISE-Bench

Metrics: accuracy relative average GEdit SC/PQ/O scores KV cache size prefill FLOPs decode latency

Baselines: FastV W-FastV PDrop VSCAN IVC-Prune

Builds on: FastV IVC-Prune VScan Flash-Attention

Closest papers by meaning

• TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.91)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.91)
• Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.90)
• Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference (ICML 2026 · similarity 0.90)
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.90)
• Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings (NeurIPS 2025 · similarity 0.90)