AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Auto-Controlled Image Perception in MLLMs via Visual Perception Tokens

Runpeng Yu, Xinyin Ma, Xinchao Wang

ICCV 2025 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Visual Perception Tokens (DINO)Qwen2-VL-2B · Average (max res 512)GPT-4o judged score: 0.572 -> 0.738Improved
add Visual Perception Tokens (DINO)Qwen2-VL-2B · Average (max res 224)GPT-4o judged score: 0.448 -> 0.576Improved
add Visual Perception Tokens (CLIP) to 7B modelQwen2-VL-7B · Average (max res 512)GPT-4o judged score: 0.624 -> 0.773Improved
allow model to freely choose perception token (Free Choice)Qwen2-VL-2B-VPT (DINO) · Average (max res 512)GPT-4o judged score: 0.738 -> 0.749Improved
use SAM instead of DINO as re-encoderQwen2-VL-2B-VPT · Average (max res 512)GPT-4o judged score: 0.738 -> 0.735No clear effect
add Visual Perception Tokens (DINO)Qwen2-VL-2B · OpenImage (max res 512)GPT-4o judged score: 0.418 -> 0.872Improved
add Visual Perception Tokens (DINO)Qwen2-VL-2B · LLaVA Instruction Tuning (max res 512)GPT-4o judged score: 0.728 -> 0.797Mixed
use region selection tokens with k=8 instead of raw bounding boxes (ablation)Region Selection Token · DocVQA (region token ablation)GPT-4o judged score: 0.219 -> 0.369Improved
increase region grid from k=8 to k=16 (ablation)Region Selection Token · DocVQA (region token ablation)GPT-4o judged score: 0.369 -> 0.307Hurt
increase region grid from k=8 to k=32 (ablation)Region Selection Token · TextVQAGPT-4o judged score: 0.686 -> 0.634Hurt
decrease region grid from k=8 to k=4 (ablation)Region Selection Token · DocVQA (region token ablation)GPT-4o judged score: 0.369 -> 0.251Hurt
add control hidden state of Re-Encoding Control token to projector (ablation)Vision Re-Encoding Token · CUB Bird (control info ablation)GPT-4o judged score: 0.7459 -> 0.8943Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy