Research map / Vision-language and multimodal / Multimodal LLMs
TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TriDF, a benchmark of about 5K real-fake pairs across 16 DeepFake types and three modalities that evaluates MLLMs on artifact perception, binary detection, and explanation hallucination.
Problem
Existing DeepFake benchmarks only provide binary labels or narrow coverage, and they do not measure whether MLLM-based detectors' explanations are grounded in real artifacts or hallucinated.
Approach
A triadic benchmark (Perception, Detection, Hallucination) built on human-annotated artifact taxonomy (quality vs. semantic) with one-to-one real-fake pairs, using an external LLM to map free-form explanations to artifact labels instead of LLM judges, and metrics Cover, CHAIR, Hal and F-beta score.
Key results
- Gemini 2.5-Pro reaches 0.7311 accuracy on Type-B image OEQ detection, the highest in that table, with Cover 0.4208.
- On TFQ perception, GPT-5 averages 60.19% and Gemini 2.5-Pro 59.63%, while Qwen3-VL-30B-Instruct reaches 59.85%; Claude Sonnet 4.5 is strongest on MCQ at 0.21 average.
- CHAIR and Hal are high for most models on Type-A OEQ, indicating hallucination is widespread, e.g. Hal above 0.95 for many open-source models on image.
Surprising findings
- Even the strongest systems only moderately outperform random choice in several perception settings.
- Localization hints do not reliably improve models' spatial focus and often introduce distractions.
Limitations
- Audio results and evaluation setup are deferred to supplementary material.
- Artifact mapping relies on an external LLM (Gemini 2.5 Flash-Lite), which may introduce its own errors.
Experimental setup
Datasets: FaceForensics++ FFHQ CelebAMaskHQ CelebA-HQ VGGFace2 Emu Edit GEdit-Bench ImgEdit OmniContext MS-COCO Flickr30k LAION-Aesthetics
Benchmarks: TriDF
Metrics: accuracy F1 Cover CHAIR Hal F0.5 score
Baselines: FakeShield FakeVLM Random Choice
Builds on: FakeBench LOKI CHAIR THRONE
Closest papers by meaning
- Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection (AAAI 2024 · similarity 0.88)
- Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector (CVPR 2025 · similarity 0.87)
- Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning (ICLR 2026 · similarity 0.87)
- A Hitchhiker's Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoning (NeurIPS 2024 · similarity 0.87)
- Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning (CVPR 2025 · similarity 0.87)
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.