Research map / Vision-language and multimodal / Multimodal LLMs

Pixels Don't Lie (But Your Detector Might): Bootstrapping MLLM-as-a-Judge for Trustworthy Deepfake Detection and Reasoning Supervision

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DeepfakeJudge, a bootstrapped generator-evaluator pipeline that trains Qwen-2.5-VL-based judges (3B/7B) to rate and compare reasoning rationales for deepfake detection, alongside an OOD benchmark and human-annotated reasoning set.

Problem

Explanations from deepfake detectors and VLMs are often ungrounded in visual evidence, and existing evaluations (classification accuracy, BLEU/ROUGE/BERTScore) do not measure reasoning fidelity.

Approach

A VLM judge trained on bootstrapped, paraphrased reasoning rationales spanning five quality levels (generator produces, evaluator scores and gives feedback, refinement until rating matches), supporting both pointwise and pairwise reasoning evaluation grounded in the image rather than ground-truth rationales.

Key results

• DeepfakeJudge-7B achieves 96.2% pairwise accuracy on DeepfakeJudge-Meta and 98.9% on DeepfakeJudge-Meta-Human.
• DeepfakeJudge-7B pointwise RMSE 0.61 and Spearman 0.94 on DeepfakeJudge-Meta; RMSE 0.50 and Pearson 0.95 on Meta-Human.
• Qwen-3-VL-235B-Instruct reaches 74.5% overall accuracy on DeepfakeJudge-Detect, the best among evaluated models, above Gemini-2.5-Flash (65.5%).

Surprising findings

• Standard text metrics (BLEU, ROUGE, METEOR, BERTScore) show low, uninformative differences across models, failing to reflect visual grounding.
• Removing paraphrasing drops DeepfakeJudge-3B pairwise accuracy from 94.4% to 89.1% and raises pointwise RMSE from 0.69 to 1.39.

Limitations

• Pairwise and pointwise evaluation uses human-verified subsets of only 100 samples per task for human agreement; small human study (10 participants, 20 samples each).
• Bootstrapping relies on GPT-4o-mini for gold rationales on real images, and generator/evaluator are the same model family, risking bias.
• Text metrics show low correlation; judge reliability is validated only against annotators, and user study significance rests on a small binomial test (18 images).

Experimental setup

Datasets: Open-Images V7 MultifakeVerse SID-SET-Description Community-Forensics DeepfakeJudge-Detect

Benchmarks: DeepfakeJudge-Detect DeepfakeJudge-Reason DeepfakeJudge-Meta DeepfakeJudge-Meta-Human

Metrics: accuracy F1 BLEU ROUGE METEOR BERTScore RMSE MSE Spearman Pearson

Baselines: Gemini-2.5-Flash GPT-4o-mini Qwen-3-VL-235B Qwen-3-VL-30B SIDA GenBuster++ BLEU ROUGE METEOR BERTScore

Builds on: VideoJudge LLM-as-a-Judge Qwen-2.5-VL

Closest papers by meaning

• PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detection (ICML 2026 · similarity 0.89)
• Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning (ICLR 2026 · similarity 0.88)
• DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models (CVPR 2026 · similarity 0.87)
• Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection (AAAI 2025 · similarity 0.86)
• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.86)
• A Hitchhiker's Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoning (NeurIPS 2024 · similarity 0.86)