Research map / Vision-language and multimodal / Human and embodied modeling

MAFT: Multimodal Automated Fact-Checking via Textualization

AAAI 2025 · Vision-language and multimodal › Human and embodied modeling · approach: Multimodal and affective interaction modeling · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MAFT, a multimodal fact-checking system that converts images, video, and audio into text (via VLM, speech recognition, and deepfake detectors), then uses LLMs with web search evidence to produce interpretable fact-checking reports.

Problem

Automated fact-checking is usually limited to specific modalities, whereas real-world multimodal misinformation combines text, images, video, and audio in arbitrary mixes.

Approach

Textualize every modality (captions/summaries via VLM, speech via Whisper, deepfake detector outputs as text) so an LLM can jointly reason over all inputs plus web-retrieved evidence, producing a structured report with claim, evidence, and verdict.

Key results

• No quantitative results are reported; the paper demonstrates the system qualitatively with a text-and-image misinformation example and a generated report (Fig. 1).

Limitations

• No quantitative evaluation or baseline comparison is provided.
• Synthetic and forged data detectors are not yet implemented (stated as future work).
• Externally retrieved multimodal information is not yet textualized as evidence.

Experimental setup

Builds on: ICT Whisper Chern et al. 2023 automatic fact-checking with LLM search queries

Closest papers by meaning

• Pioneering Explainable Video Fact-Checking with a New Dataset and Multi-role Multimodal Model Approach (AAAI 2025 · similarity 0.85)
• DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts (ICML 2025 · similarity 0.84)
• Multimodal Fact-Level Attribution for Verifiable Reasoning (ICML 2026 · similarity 0.84)
• Beyond Text: Fine-Grained Multi-Modal Fact Verification with Hypergraph Transformers (AAAI 2025 · similarity 0.83)
• MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs (ICLR 2025 · similarity 0.82)
• AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web (NeurIPS 2025 · similarity 0.82)