Research map / Vision-language and multimodal / Multimodal LLMs

Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PCGR, an interpretable multimodal misinformation detector that automatically grows a layered probabilistic concept graph with MLLM-proposed concepts and infers veracity via top-down hierarchical attention.

Problem

Multimodal misinformation detectors are opaque black boxes that are fragile to new manipulation tactics and domain shift, and existing interpretable variants rely on fixed concept sets or post-hoc rationales.

Approach

Reframes MMD as concept-based probabilistic reasoning: MLLMs iteratively propose and filter new interrogative concepts, which form a layered DAG of soft predicates; veracity is computed by top-down multiplicative attention aggregation over the concept graph.

Key results

• Coarse-level accuracy of 80.2% on MiRAGeNews, 80.6% on MMFakeBench, and 84.3% on AMG, versus 72.3%, 74.1%, 78.2% for the strongest baseline MGCA.
• On fine-grained AMG, PCGR reaches 75.6 Mic-F1 and 59.9 Mac-F1 versus MGCA's 72.8 and 57.2.
• Ablation on AMG: removing automatic concept generation (w/o acg) causes the largest drop, about 12.9% Mic-F1 and 12.5% Mac-F1.

Surprising findings

• General-purpose MLLMs such as GPT-5 perform far below dedicated multimodal detectors on binary and fine-grained MMD, with LLaVA-1.6 near chance accuracy.

Limitations

• Concept generation depends on proprietary GPT-5 API, limiting reproducibility and adding cost.
• Several equations and notation are inconsistent in the text (e.g., q_k vs p_k, undefined t_i vs h_i, NLI outputs mismatched), making the method hard to reproduce exactly.
• Performance is evaluated only on three benchmarks, and the hyperparameter thresholds (e.g., max 5 concepts per layer, 6 layers) were tuned on the same evaluation setup.

Experimental setup

Datasets: MiRAGeNews MMFakeBench AMG

Benchmarks: MiRAGeNews MMFakeBench AMG

Metrics: Accuracy Precision Recall F1 Micro-F1 Macro-F1

Baselines: LLaVA-1.6 Qwen3-Omni Llama 4 Gemini 2.5 Pro GPT-5 MPFN BMR HAMMER MiRAGe FKA-Owl GLPN-LLM C3N

Builds on: Concept Bottleneck Models Graph-of-Thought prompting CLIP Sentence-BERT

Closest papers by meaning

• CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection (ICML 2026 · similarity 0.88)
• SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection (CVPR 2024 · similarity 0.86)
• Beyond Text: Fine-Grained Multi-Modal Fact Verification with Hypergraph Transformers (AAAI 2025 · similarity 0.84)
• DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs (CVPR 2026 · similarity 0.84)
• ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection (NeurIPS 2025 · similarity 0.84)
• TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning (NeurIPS 2026 · similarity 0.84)