Research map / Video / Video generation
SynthForensics: Benchmarking and Evaluating People-Centric Synthetic Video Deepfakes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SynthForensics, a 20,445-video people-centric benchmark from 8 T2V and 7 I2V open-source DiT-era generators paired with FF++/DFD real sources, and evaluates 15 deepfake detectors zero-shot, after fine-tuning, and when trained from scratch.
Problem
Existing deepfake benchmarks target manipulation-based forgeries or mix heterogeneous non-human content, so detectors are not evaluated on realistic people-centric purely synthetic videos from modern T2V/I2V generators.
Approach
A paired-source benchmark where captions of real FF++/DFD videos drive T2V and I2V generation from recent open-source DiT and autoregressive generators, with two-stage human validation, four compression versions, and metadata, plus human paired-comparison and landmark/quality analyses against nine existing synthetic-video benchmarks.
Key results
- Face-based zero-shot detectors lose on average 27 AUC points from FF++ to SF-FF++; FTCN drops ~55 points and LAA-Net falls to 47% on SF-CDF.
- In human paired comparisons, SynthForensics wins 71.5% on quality and 76.5% on realism against existing benchmarks, and is flagged as fake 37.5% vs 84.1% for competing videos.
- Fine-tuning lifts SF-FF++ AUC to 96-99% for most detectors but AltFreezing and FTCN drop to 65.5% and 57.1% on FF++.
Surprising findings
- Training from scratch on synthetic videos yields legacy transfer collapsing to 50-65% AUC for most detectors, with DFD-FCG the exception at 83.4%.
- Purpose-built synthetic-video detectors (MM-Det, NSG-VD, D3) perform poorly, with AUC between 41% and 71%, often near chance.
Limitations
- Only open-source generators are included; proprietary generators are excluded, so results may not cover the closed-source state of the art.
- Source data is limited to 1,363 FF++ and DFD videos; some detectors are excluded from fine-tuning or training due to missing training code or required supervision.
- Human study relies on remote, uncontrolled participation with a relatively small number of participants (118 retained).
Experimental setup
Datasets: FaceForensics++ Deep Fake Detection (DFD) Celeb-DF (CDF) GenVideo GenVidBench AIGVDBench DVF GVF GVD LOKI AEGIS DeepAction
Benchmarks: SF-FF++ SF-DFD SF-CDF FF++ DFD CDF
Metrics: AUC landmark completeness TOPIQ-NR-Face score win rate fake-flag rate Krippendorff's alpha
Baselines: CFM RECCE ProDet UCF Effort LAA-Net GenD AltFreezing FTCN GenConViT DFD-FCG FakeSTormer
Builds on: FaceForensics++ DeepfakeBench HunyuanVideo structured captioning TOPIQ 2DFAN4
Closest papers by meaning
- Video-Bench: Human-Aligned Video Generation Benchmark (CVPR 2025 · similarity 0.87)
- VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis (CVPR 2025 · similarity 0.86)
- OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation (CVPR 2025 · similarity 0.86)
- Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Content (CVPR 2025 · similarity 0.86)
- VBench: Comprehensive Benchmark Suite for Video Generative Models (CVPR 2024 · similarity 0.86)
- TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.