Research map / Vision-language and multimodal / Multimodal LLMs

VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VisionThink, an RL-trained VLM that starts from a 1/4-resolution image and learns to call a resize tool to request full resolution only when needed, using an LLM-as-Judge reward for general VQA.

Problem

VLMs consume many visual tokens for every input, yet most general VQA questions can be answered from low-resolution images while OCR-heavy tasks need high resolution; fixed token-pruning ratios ignore this per-sample variation, and rule-based RL does not transfer to open-ended general VQA.

Approach

Let the VLM decide per sample whether to answer from a downsampled image or request a 2x higher-resolution image via a resize tool call, trained with multi-turn GRPO, an LLM-as-Judge accuracy reward for general VQA, and a threshold-based penalty that prevents collapse to always-resize or never-resize.

Key results

• VisionThink retains about 51.3% of visual tokens and reaches an average of 101.4% of vanilla Qwen2.5-VL-7B performance across nine benchmarks, versus 95.8% for FastV at 50% retention.
• On MMVet and MathVerse, VisionThink scores 67.1 and 48.0, improvements of 8.9% and 3.7% over the base model; MME reaches 2400.
• On DocVQA, VisionThink's inference is more than twice as fast as QwenRL at full resolution.

Measured effects reported in the paper

Surprising findings

• Without a penalty on resize calls, the model collapses to always requesting high resolution; a naive fixed penalty instead collapses it to always answering directly due to lucky guesses on blurry images.

Limitations

• Only 2x resolution upscaling and at most two turns of interaction are explored.
• Only image resizing is used as the visual tool; cropping and other tools are not explored.
• ChartQA evaluation via lmms-eval float matching is flawed, so the paper uses GPT-4o judge plus human verification, making comparisons non-standard.

Experimental setup

Datasets: 10K high-resolution-required VQA samples 10K low-resolution-solvable VQA samples 130K filtered general VQA SFT-derived QA pairs

Benchmarks: MMMU MMMU-Pro MMBench RealWorldQA POPE MME MathVista MathVerse MMVet ChartQA OCRBench DocVQA

Metrics: accuracy F1 (POPE) relative score vs vanilla inference time (vLLM) fraction of visual tokens retained

Baselines: FastV SparseVLM Qwen2.5-VL-7B-Instruct QwenRL GPT-4o InternVL2.5-8B Vision-R1-LlamaV-CI-11B

Builds on: GRPO Search-R1 Qwen2.5-VL veRL DAPO

Closest papers by meaning

• Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters (ICLR 2025 · similarity 0.90)
• Rethinking Token Reduction for Large Vision-Language Models (CVPR 2026 · similarity 0.90)
• ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.89)
• AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition (CVPR 2026 · similarity 0.89)
• SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference (ICML 2025 · similarity 0.89)
• Hybrid Token Compression for Vision-Language Models (CVPR 2026 · similarity 0.89)