Research map / Vision-language and multimodal / Multimodal LLMs

ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ViCO, a two-stage training method (KL-based consistency training across multiple MLP connectors with different compression ratios, then a patch-level Visual Resolution Router) that cuts InternVL3.5 vision tokens by up to ~50-80% per patch while retaining ~99.6% of performance.

Problem

MLLMs incur high inference cost from the many vision tokens produced by dynamic high-resolution image splitting, especially for multi-image and document inputs.

Approach

Train multiple MLP connectors with different downsampling ratios so the model's responses under compressed and uncompressed visual tokens are consistent (KL to a frozen reference), then train a lightweight patch-level router that predicts which patches can be safely compressed, using a loss-ratio pseudo-label.

Key results

• InternVL3.5-8B with ViCO retains 99.6% average performance across general and OCR benchmarks with about 69% of patches compressed (average ratio 69.1%).
• On the ablation study with InternVL3.5-8B, full ViCO reaches 99.6% retention with 55% average compressed patches, versus 97.6% for random routing and 95.1% for uniformly compressing all patches.
• LLM first-token throughput improves about 1.97x at 50% token compression with 64 text tokens across model scales (8B, 30B, 38B, 241B).

Surprising findings

• Directly applying the trained ViR to the original InternVL3.5 without consistency training drops OCRBench from 84.0 to 75.1, showing the base model cannot handle interleaved compression rates.
• Image-level routing leads to highly inconsistent compression across datasets (12% compressed on InfoVQA vs 88% on BLINK).

Limitations

• Ablation and comparison experiments restricted to InternVL3.5-8B; GPU hours and training data scale not reported.
• Some benchmarks show drops (e.g., MMMU 73.4 to 71.9 at 8B, RealWorldQA drops at 38B), and the paper's claims of matching performance are approximate.
• Code and models are only promised for future release.

Experimental setup

Benchmarks: OCRBench ChartQA TextVQA MMStar POPE MME MMBench V1.1 RealWorldQA MMMU BLINK MMT-Bench MMIU

Metrics: accuracy performance retention (%) ratio of compressed patches first-token throughput speedup

Baselines: FastV SparseVLM InternVL3.5-4B InternVL3.5-8B InternVL3.5-14B InternVL3.5-30B-MoE InternVL3.5-38B InternVL3.5-241B-MoE

Builds on: InternVL3.5 dynamic high-resolution FastV SparseVLM

Closest papers by meaning

• Conical Visual Concentration for Efficient Large Vision-Language Models (CVPR 2025 · similarity 0.90)
• Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings (NeurIPS 2025 · similarity 0.89)
• Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.89)
• Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models (ICLR 2025 · similarity 0.89)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.88)
• EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling (CVPR 2026 · similarity 0.88)