Research map / Video / Video segmentation and tracking
Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes video-level blending data (VB) simulating facial feature drift and a lightweight plug-and-play spatiotemporal adapter (StA) on frozen image backbones for generalizable deepfake video detection.
Problem
Deepfake video detectors generalize poorly across datasets, lack general temporal artifacts, tend to learn only spatial or only temporal cues, and are computationally expensive.
Approach
Simulates a temporal forgery artifact (Facial Feature Drift) by frame-by-frame warping and blending facial organ regions as hard negatives, and adds lightweight two-stream depthwise 3D-conv adapters (spatial and temporal with multi-scale kernels) with cross-attention into a frozen pretrained image model.
Key results
- Cross-dataset average AUC of 0.911 across seven image-detector benchmarks in Protocol-1, versus 0.856 for the next-best CFM.
- With CLIP ViT-L/14, the method reaches 0.947 AUC on Celeb-DF-v2 and 0.843 on DFDC using 46M trainable parameters versus 307M for the backbone.
- Cross-attention fusion improves average AUC from 0.863 to 0.905 over linear combination in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply StA and VB to CLIP ViT-L14 | CLIP ViT-L14 · Cross-dataset average | AUC: 0.854 -> 0.895 | Helped |
| add StA without VB (ablation) | CLIP ViT-L14 · CDF-v2 | AUC: 0.900 -> 0.926 | Helped |
| add StA without VB (ablation) | CLIP ViT-L14 · WDF | AUC: 0.794 -> 0.842 | Helped |
| add VB to StA (ablation) | CLIP ViT-L14 + StA · CDF-v2 | AUC: 0.926 -> 0.947 | Helped |
| add VB to StA (ablation) | CLIP ViT-L14 + StA · WDF | AUC: 0.842 -> 0.848 | Helped |
| add VB to StA (ablation) | CLIP ViT-L14 + StA · CDF-v2 | accuracy: 0.812 -> 0.838 | Helped |
| replace temporal VB with fixed image-level SBI blending (ablation) | video detector · CDF-v2 | AUC: 0.947 -> 0.902 | Hurt |
| replace temporal VB with fixed image-level SBI blending (ablation) | video detector · DFDC | AUC: 0.843 -> 0.815 | Hurt |
| combine fixed SBI with VB (ablation) | video detector · CDF-v2 | AUC: 0.947 -> 0.951 | Helped |
| combine fixed SBI with VB (ablation) | video detector · DFDC | AUC: 0.843 -> 0.846 | Helped |
| replace cross-attention fusion with linear addition (ablation) | StA · CDF-v2 | AUC: 0.947 -> 0.886 | Hurt |
| replace cross-attention fusion with linear addition (ablation) | StA · DFDC | AUC: 0.843 -> 0.787 | Hurt |
Surprising findings
- Image-level blending (fixed SBI) alone is weaker than video-level blending for temporal artifacts, yet combining both gives further gains.
Limitations
- Relies on facial landmarks and region masks, limiting applicability to non-face manipulations.
- Some competing methods' results are cited from original papers, and DFDC is no longer downloadable, complicating comparison.
Experimental setup
Datasets: FaceForensics++ c23 DF40
Benchmarks: Celeb-DF-v2 DFD DFDC DFDCP DeeperForensics WildDeepfake FFIW DF40 FaceForensics++
Metrics: AUC accuracy EER
Baselines: SBI UCF RECCE LSDA CFM LAA-Net AltFreezing NACO RealForensics VideoMAE
Builds on: SBI Face X-ray AltFreezing CLIP DeepfakeBench
Closest papers by meaning
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.90)
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.89)
- Exploiting Style Latent Flows for Generalizing Deepfake Video Detection (CVPR 2024 · similarity 0.89)
- On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection (NeurIPS 2024 · similarity 0.89)
- Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection (AAAI 2024 · similarity 0.89)
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.