AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

BLINK: Multimodal Large Language Models Can See but Not Perceive

Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, Ranjay Krishna

ECCV 2024 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 349 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace images with dense captions and use text-only GPT-4 (Caption + LLM) (ablation)GPT-4V · MMBenchaccuracy: +5.7% accuracy over GPT-4VImproved
replace images with dense captions and use text-only GPT-4 (Caption + LLM) (ablation)GPT-4V · MMMUaccuracy: 47.2% vs 9.6% lower than GPT-4VHurt
replace images with dense captions and use text-only GPT-4 (Caption + LLM) (ablation)GPT-4V · BLINKaccuracyHurt
use red circles instead of gray circles as visual prompt (ablation)GPT-4V · BLINK depth, reflectance, visual correspondence (100 dev samples each)accuracyImproved
vary visual prompt circle radius (5, 10, 15 px) (ablation)GPT-4V · BLINK depth, reflectance, visual correspondenceaccuracyMixed
use specialist vision models instead of multimodal LLMsBLINK tasks · BLINK dev, visual correspondenceaccuracy: 33.72 -> 96.51Improved
use specialist DepthAnything instead of GPT-4VBLINK relative depth · BLINK devaccuracy: 59.68 -> 97.58Improved
scale open-source models to 7B/13Bopen-source multimodal LLMs · BLINK testaccuracy: 35-42% vs random 38.09%No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy