Research map / Vision-language and multimodal / Multimodal LLMs

Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MMKC-Bench, a benchmark of 1,573 knowledge instances and 3,381 images testing multimodal factual knowledge conflicts (context-memory and inter-context) plus conflict detection in LMMs, finding models favor parametric knowledge over external evidence.

Problem

Existing multimodal knowledge conflict benchmarks focus on intra-memory conflicts and observe behavior without testing whether LMMs detect conflicts, leaving context-memory and inter-context factual conflicts under RAG underexplored.

Approach

Builds a benchmark of counterfactually edited multimodal knowledge (entity name, visual semantic meaning, or factual attribute) with two conflict settings (one or two conflicting evidence pieces) and both behavior analysis and coarse/fine-grained conflict detection tasks.

Key results

• Under context-memory conflict, average OAR exceeds CAR for all evaluated models (e.g., Qwen2.5-VL-7B MCQ OAR 0.67 vs CAR 0.30), showing preference for parametric knowledge.
• Average conflict detection accuracy reaches 79% (Qwen2.5-VL-7B), 75% (InternVL3-8B), and 76% (GPT-4o mini) in coarse-grained detection.
• Inter-context conflicts raise CAR only modestly over context-memory conflicts, with limited gains on average.

Surprising findings

• Unlike LLMs known to be receptive to external context, LMMs including GPT-4o mini favor internal parametric knowledge over conflicting external evidence.
• Entity recognition conflicts show the highest OAR, meaning models more readily rely on internal knowledge for perception-style conflicts.

Limitations

• Conflicts are synthesized via counterfactual editing, creating a distribution gap from naturally occurring real-world conflicts.
• Fine-grained conflict detection is slightly lower than coarse-grained detection.

Experimental setup

Datasets: Wikipedia Google Images MMKE

Benchmarks: MMKC-Bench

Metrics: OAR CAR IAR detection accuracy

Baselines: Qwen2.5-VL InternVL3 GPT-4o mini

Builds on: MMKE ConflictBank WikiConflict

Closest papers by meaning

• CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution (AAAI 2026 · similarity 0.89)
• $\texttt{ConflictBank}$: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMs (NeurIPS 2024 · similarity 0.89)
• Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (ICLR 2024 · similarity 0.87)
• ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge (NeurIPS 2026 · similarity 0.87)
• CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection (ICML 2026 · similarity 0.87)
• WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia (NeurIPS 2024 · similarity 0.87)