Research map / Video / Video segmentation and tracking
Exploiting Style Latent Flows for Generalizing Deepfake Video Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StyleGRU, which models temporal changes (style flows) of pSp StyleGAN style latent vectors with contrastive and classification losses, fused with 3D ResNet-50 content features via a style attention module for generalizable deepfake video detection.
Problem
Deepfake video detectors trained on FF++ generalize poorly to newer high-quality generators that suppress visual and temporal artifacts, so cross-dataset and cross-manipulation generalization remains weak.
Approach
Use the temporal differences (style flows) of StyleGAN style latent vectors extracted by a pSp encoder, encoded by a bidirectional GRU trained with triplet and BCE losses, and fuse them with 3D CNN content features through a style attention module (query from content, key/value from style flow) before a temporal transformer classifier.
Key results
- Achieves 95.8 average AUC across CDF, DFD, FSh, and DFo cross-dataset evaluation, the best average versus AltFreeze at 95.2.
- Reaches 98.4 average video-level AUC on cross-manipulation over DF, FS, F2F, NT, vs AltFreeze 98.6 and FTCN 97.5.
- StyleGRU alone yields 79.8 average cross-dataset AUC; removing SAM drops CDF/FSh average to 93.4 vs 94.0 full (ablation table).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add StyleGRU style-flow features to the model (ablation) | Deepfake detector · CDF | AUC (%): 85.7 -> 89.0 | Helped |
| Add StyleGRU style-flow features to the model (ablation) | Deepfake detector · FSh | AUC (%): 98.4 -> 99.0 | Helped |
| Add the Style Attention Module (SAM) to fuse style and content features (ablation) | Deepfake detector · CDF | AUC (%): 88.4 -> 89.0 | Helped |
| Add the Style Attention Module (SAM) to fuse style and content features (ablation) | Deepfake detector · FSh | AUC (%): 98.4 -> 99.0 | Helped |
| Use differences between consecutive style latent vectors as input (ablation) | StyleGRU deepfake detector · CDF | AUC (%): 87.8 -> 89.0 | Helped |
| Use differences between consecutive style latent vectors as input (ablation) | StyleGRU deepfake detector · FSh | AUC (%): 98.3 -> 99.0 | Helped |
| Add triplet loss to classification loss during StyleGRU training (ablation) | StyleGRU · CDF | AUC: 88.6 -> 89.0 | Helped |
| Add triplet loss to classification loss during StyleGRU training (ablation) | StyleGRU · FSh | AUC: 99.0 -> 99.0 | No clear effect |
| Use both classification loss and triplet loss instead of classification loss alone (ablation) | StyleGRU · FF++ | AUC: 99.2 -> 99.1 | Hurt |
| Use triplet loss without classification loss (ablation) | StyleGRU · CDF | AUC: 89.0 -> 88.4 | Hurt |
| Use triplet loss without classification loss (ablation) | StyleGRU · FSh | AUC: 99.0 -> 98.8 | Hurt |
Surprising findings
- The StyleGRU-only classifier performs poorly on Celeb-DF (62.5 AUC) despite strong FaceShifter performance (90.7).
- On cross-manipulation, the proposed method is slightly below AltFreeze on average (98.4 vs 98.6).
Limitations
- Preprocessing via pSp style-vector extraction is time-consuming.
- Method focuses only on facial attributes from StyleGAN; generalization to other object categories is untested.
- Performance under some perturbations (e.g., resize, blur) is slightly deficient due to pSp not being noise-robust.
Experimental setup
Datasets: FaceForensics++ c23
Benchmarks: Celeb-DF-v2 DFD FaceShifter DeeperForensics FaceForensics++ cross-manipulation
Metrics: AUC
Baselines: Xception CNN-aug PatchForensics Face X-ray SBI-R50 CNN-GRU LipForensics RealForensics FTCN AltFreeze
Builds on: FTCN pSp AltFreeze supervised contrastive learning
Closest papers by meaning
- Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning (CVPR 2025 · similarity 0.89)
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.87)
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.87)
- Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Content (CVPR 2025 · similarity 0.86)
- Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing (AAAI 2025 · similarity 0.86)
- Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.