Research map / Vision-language and multimodal / Multimodal LLMs

Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VI-Probe, a controllable visual-illusion benchmark with graded perturbations and matched controls, plus paired-prompt metrics (PFC, TFI, CbW) and an illusion multiplier R, to diagnose whether VLMs perceive visual change or recall memorized illusion answers.

Problem

VLMs answer classic visual illusions correctly on original images but keep the same responses when illusion factors are inverted, and it is unclear whether this reflects visual perception or memorized patterns; average accuracy hides this gap.

Approach

A controllable illusion generator produces original, perturbed (graded factor inversion), matched control (inducer removed), and hinted images, paired with forward/reversed/instructional questions; metrics PFC, TFI, CbW and illusion multiplier R (illusion drop normalized by control drop) separate memory-driven from perception-limited behavior.

Key results

• GPT-5 reaches 91.72% accuracy on original illusion images but drops to 4.45% on perturbed illusions, with illusion multiplier R=1.97, indicating memory override.
• Claude-Opus-4.1 and Claude-Sonnet-4.5 show R≈1.01, interpreted as perception-memory competition.
• Qwen2.5-VL-3B has TFI of 46.82% and R=0.13, interpreted as visual-processing limits; Qwen models uniformly show R<1.

Surprising findings

• Scaling does not monotonically reduce illusion susceptibility; Qwen3-VL-8B has higher PFC than Qwen3-VL-32B, and within families smaller models sometimes outperform larger ones.
• Visual hints improve original-illusion accuracy for most models but degrade perturbed accuracy for 12 of 15 models.

Limitations

• Benchmark covers only 27 classic illusions focused on size, color, and orientation.
• Evaluation relies on API-accessed models with default settings for some, limiting controlled comparison.

Experimental setup

Benchmarks: VI-Probe

Metrics: PFC PFA TFI CbW illusion multiplier R accuracy

Baselines: HallusionBench IllusionBench+ IllusionVQA VLMBiased

Builds on: HallusionBench IllusionVQA

Closest papers by meaning

• Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination (ICML 2026 · similarity 0.89)
• Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy (ICML 2026 · similarity 0.88)
• In-Context Learning Can Help Vision Language Models Overcome Training Prior (NeurIPS 2026 · similarity 0.87)
• VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding (NeurIPS 2025 · similarity 0.87)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.87)
• VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models (CVPR 2026 · similarity 0.87)