Research map / Vision-language and multimodal / Multimodal LLMs

From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BridgeVLM, which induces a latent causal DAG from multi-image inputs inside a Phi-4-MMI-7B VLM, routes information via RAMP layers into Causal Tokens, and supervises them with M3S to improve multi-image interventional and counterfactual reasoning.

Problem

Vision-language models are brittle on interventional-target and counterfactual causal reasoning over multi-image inputs, and prompt-level causal knowledge (text graphs, rationales) has little effect, especially when causal supervision is missing or heterogeneous.

Approach

Induce a routing DAG over learnable variable features from visual tokens, propagate information along it with route-aware message passing (RAMP), and expose the resulting node and graph tokens to the LLM decoder as Causal Tokens, supervised directly (not via prompt text) by a unified M3S loss that accepts any subset of graph edges, node/global descriptions, and acyclicity regularization.

Key results

• BridgeVLM-7B reaches 54.4% average intervention accuracy on CausalVLBench vs 33.2% for Phi-4-MMI-7B with causal-trace prompt supervision (Gemini-2.0-Flash reports 56.4%).
• On Causal3D counterfactual accuracy rises to 92.3% from 81.0% for the prompt-supervised Phi-4 baseline.
• Directed-edge F1 for the induced DAG on CausalVLBench improves from 33.4% (answer-only) to 75.1% with NodeAlign.

Surprising findings

• Ground-truth graph supervision on the DAG alone (GraphAlign) yields no gain or hurts intervention accuracy (31.0 avg) without node-level alignment, attributed to variable-to-node misalignment.
• Zero-shot transfer from CausalVLBench to CELLO is worse than the untuned Phi-4 baseline (63.1% vs 66.2%).

Limitations

• The induced DAG is a latent routing structure and is not guaranteed to recover the true causal graph without assumptions.
• Generalization to unseen causal scenarios or variable sets is not demonstrated; zero-shot OOD transfer underperforms.
• Adds about 0.8B parameters and roughly 1.8x inference latency over the backbone.

Experimental setup

Datasets: CausalVLBench Causal3D CELLO

Benchmarks: CausalVLBench Causal3D CELLO

Metrics: accuracy directed-edge F1

Baselines: LLaVA-OneVision-7B DeepSeek-VL2-S-16B Qwen2.5-VL-32B Gemini-2.0-Flash Phi-4-MMI-7B (answer) Phi-4-MMI-7B (causal-trace) Slot-Attention

Builds on: Phi-4-multimodal NOTEARS Slot Attention DETR Hungarian matching

Closest papers by meaning

• The Abstraction Gap in Vision-Language Causal Reasoning (ICML 2026 · similarity 0.91)
• Vision-and-Language Navigation via Causal Learning (CVPR 2024 · similarity 0.88)
• CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulations (NeurIPS 2025 · similarity 0.88)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.88)
• Causal Graphical Models for Vision-Language Compositional Understanding (ICLR 2025 · similarity 0.88)
• Visual Representation Learning through Causal Intervention for Controllable Image Editing (CVPR 2025 · similarity 0.88)