Research map / Vision-language and multimodal / Multimodal LLMs

AbductiveMLLM: Boosting Visual Abductive Reasoning Within MLLMs

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AbductiveMLLM, which pairs a causality-aware hypothesis selector (Reasoner) with a Stable Diffusion-based pictorial imaginer (Imaginer) to improve visual abductive reasoning in Qwen2VL-7B, trained end-to-end.

Problem

MLLMs lag behind humans on visual abductive reasoning (VAR), i.e., inferring the most likely explanation for incomplete visual event sequences.

Approach

Combines verbal abduction (LLM-generated hypotheses pruned by a cross-modal causal contrastive model and fed to an MLLM as priors) with pictorial abduction (a diffusion model with lightweight spatiotemporal adapters conditioned on video and MLLM embeddings that "imagines" the scene), trained jointly end-to-end.

Key results

• On VAR test, AbductiveMLLM reaches CIDEr 57.04 and ROUGE 27.95, versus 50.82 CIDEr and 27.11 ROUGE for the fine-tuned Qwen2VL-7B baseline.
• Compared with the best traditional model UPD-Trans on VAR, it improves BLEU@4 by +1.14 and CIDEr by +15.38.
• On YouCookII test, it achieves CIDEr 77.70 versus 68.44 for Qwen2VL-7B-FT.

Limitations

• Gains over the fine-tuned Qwen2VL-7B baseline are modest (e.g., +0.84 ROUGE on VAR), and a large gap to human performance remains.
• Hypothesis generation depends on closed-source GPT-4o-mini and on captions from a pre-trained MLLM.

Experimental setup

Datasets: VAR YouCookII

Benchmarks: VAR YouCookII

Metrics: BLEU@4 METEOR ROUGE-L CIDEr BERT-S

Baselines: REASONER UPD-Trans KN-VLM GPT-4o-mini VideoChat2-7B Qwen2VL-7B Qwen2VL-7B-FT

Builds on: Stable Diffusion Qwen2-VL NT-Xent Min-SNR weighting

Closest papers by meaning

• NL-Eye: Abductive NLI For Images (ICLR 2025 · similarity 0.91)
• Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (CVPR 2026 · similarity 0.90)
• Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Events (CVPR 2025 · similarity 0.90)
• VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning (ICLR 2025 · similarity 0.89)
• Look-Back: Implicit Visual Re-focusing in MLLM Reasoning (AAAI 2026 · similarity 0.89)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.89)