Research map / Vision-language and multimodal / Multimodal LLMs

The Abstraction Gap in Vision-Language Causal Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces the CAGE benchmark and a dual-probe methodology with an Abstraction Gap metric to show that most VLMs write fluent causal text but fail to generate explicit causal chains, and that chain-annotated fine-tuning does not close the gap for low-baseline models.

Problem

Current evaluations of vision-language models cannot distinguish linguistic plausibility of causal explanations from faithful causal reasoning, especially whether models can generate causal structure before text.

Approach

A dual-probe evaluation (Text-Only vs. Chain-Text, where a minimal arrow-notation causal chain is generated before text) plus a normalized Abstraction Gap metric that measures the drop from text score to chain score on COCO-based causal questions across Pearl's hierarchy.

Key results

• Seven of eight evaluated VLMs have Abstraction Gap above 0.50 (e.g., mPLUG-Owl2 0.92, CogVLM2 0.88) while text scores are 6-8 out of 10.
• Fine-tuning on 45,000 chain-annotated examples does not close the gap for low-baseline models; MiniGPT-4 chain scores drop at L3 (2.05 to 0.54).
• Gemini 2.5 Flash shows AG of -0.006 (text 8.60, chain 8.46), while Qwen3-VL shows AG 0.313.

Surprising findings

• Chain-supervised fine-tuning does not improve chain generation (F+C vs F-C nearly identical), and MiniGPT-4 chain scores decrease after fine-tuning.
• Fine-tuning on causal chains does not change hallucination rates, suggesting perceptual and structural grounding are independent.

Limitations

• CAGE evaluates only linear causal chains without branching, cycles, or confounders.
• Uses COCO images, so generalization to specialized domains such as medical imaging is untested.
• Causal explanations for RLHF suppression and MoE architecture are hypotheses not directly tested.

Experimental setup

Datasets: COCO 2017 CAGE

Benchmarks: CAGE POPE MMHal-Bench

Metrics: mean score (0-10) Abstraction Gap bootstrap 95% CI Wilcoxon signed-rank p-value POPE accuracy MMHal hallucination rate

Baselines: LLaVA-NeXT LLaVA-RLHF CogVLM2 MiniGPT-4 mPLUG-Owl2 Qwen-VL-Chat InternVL2 LLaVA-CoT Qwen3-VL Gemini 2.5 Flash

Builds on: Pearl's causal hierarchy chain-of-thought prompting LIMO

Closest papers by meaning

• From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning (ICML 2026 · similarity 0.91)
• Think Visually, Reason Textually: Vision-Language Synergy in Abstract Reasoning (CVPR 2026 · similarity 0.90)
• When to Think and When to Look: Uncertainty-Guided Lookback (CVPR 2026 · similarity 0.89)
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging (ICML 2025 · similarity 0.89)
• Mitigating Hallucinations in Large Language Models via Causal Reasoning (AAAI 2026 · similarity 0.89)
• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.89)