Research map / Vision-language and multimodal / Multimodal LLMs
BLINK: Multimodal Large Language Models Can See but Not Perceive
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, Ranjay Krishna
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace images with dense captions and use text-only GPT-4 (Caption + LLM) (ablation) | GPT-4V · MMBench | accuracy: +5.7% accuracy over GPT-4V | Improved |
| replace images with dense captions and use text-only GPT-4 (Caption + LLM) (ablation) | GPT-4V · MMMU | accuracy: 47.2% vs 9.6% lower than GPT-4V | Hurt |
| replace images with dense captions and use text-only GPT-4 (Caption + LLM) (ablation) | GPT-4V · BLINK | accuracy | Hurt |
| use red circles instead of gray circles as visual prompt (ablation) | GPT-4V · BLINK depth, reflectance, visual correspondence (100 dev samples each) | accuracy | Improved |
| vary visual prompt circle radius (5, 10, 15 px) (ablation) | GPT-4V · BLINK depth, reflectance, visual correspondence | accuracy | Mixed |
| use specialist vision models instead of multimodal LLMs | BLINK tasks · BLINK dev, visual correspondence | accuracy: 33.72 -> 96.51 | Improved |
| use specialist DepthAnything instead of GPT-4V | BLINK relative depth · BLINK dev | accuracy: 59.68 -> 97.58 | Improved |
| scale open-source models to 7B/13B | open-source multimodal LLMs · BLINK test | accuracy: 35-42% vs random 38.09% | No clear effect |
Closest papers by meaning
- Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs (CVPR 2024 · similarity 0.91)
- Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs (CVPR 2026 · similarity 0.91)
- Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.91)
- Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.91)
- Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.90)
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
