Research map / Video / Video segmentation and tracking
Face Forgery Video Detection via Temporal Forgery Cue Unraveling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TFCU, a framework that unravels temporal forgery cues (momentary anomaly, gradual inconsistency, cumulative distortion) via consecutive correlate, future guide, and historical review modules for generalizable face forgery video detection.
Problem
Face forgery video detection generalization to unseen datasets and manipulation methods; existing methods coarsely exploit temporal information, leaving inter-frame prediction fluctuations.
Approach
Progressively unravel temporal forgery cues across three levels: a Transformer over consecutive frames (with random masking) captures momentary anomalies; iterative aggregation and landmark-prompted propagation of these cues into future clips captures gradual inconsistencies; and a momentum-based post-hoc accumulation from future to historical frames captures cumulative distortions.
Key results
- TFCU reaches 91.38% frame-wise AUC on Celeb-DF, 85.43% on DFDC and 90.21% on FFIW in cross-dataset evaluation, the best among compared methods.
- On cross-manipulation DF40 tests, TFCU gets 91.16% frame-wise AUC on SadTalker and 99.21% on FOMM, improving SadTalker frame-wise AUC by 4.50% over the best prior method.
- Ablation: the consecutive correlate, future guide, and historical review modules raise average frame-wise AUC from 82.85 (baseline) to 85.55, 87.68, and 89.01 (full model with 4 frames per clip).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the consecutive correlate module (CCM) to the real-face-pretrained ViT baseline (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 83.41 -> 86.30; 80.73 -> 82.99; 84.40 -> 87.36 | Helped |
| Add random sampling (RS) for variable-length consecutive frames (ablation) | TFCU with CCM · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 86.30 -> 84.57; 82.99 -> 82.45; 87.36 -> 86.54 | Helped |
| Add random masking (RM) to CCM with RS (ablation) | TFCU with CCM and RS · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 84.57 -> 86.30; 82.45 -> 82.99; 86.54 -> 87.36 | Helped |
| Add the future guide module (FGM) (ablation) | TFCU with baseline and CCM · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 86.30 -> 89.52; 82.99 -> 83.83; 87.36 -> 88.20 | Helped |
| Add forgery feature aggregation (FFA) (ablation) | TFCU with FGM · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 89.52 -> 89.75; 83.83 -> 84.08; 88.20 -> 88.41 | Helped |
| Add landmark prompt encoder (LPE) (ablation) | TFCU with FGM and FFA · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 89.75 -> 89.75; 84.08 -> 84.34; 88.41 -> 88.96 | Helped |
| Add historical review module (HRM) with inverse linear decay (ILDF) (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 89.75 -> 91.38; 84.34 -> 85.43; 88.96 -> 90.21 | Helped |
| Use logarithmic decay (LDF) instead of inverse linear decay (ILDF) in HRM (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 89.75 -> 91.27; 84.34 -> 85.44; 88.96 -> 90.14 | Helped |
| Use exponential decay (EDF) instead of inverse linear decay (ILDF) in HRM (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 89.75 -> 89.80; 84.34 -> 84.47; 88.96 -> 89.06 | No clear effect |
| Use 4 rather than 2 frames per clip (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 89.81 -> 91.38; 84.03 -> 85.43; 89.56 -> 90.21 | Helped |
| Use 6 rather than 4 frames per clip (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 91.38 -> 91.17; 85.43 -> 84.42; 90.21 -> 90.29 | Hurt |
| Use 8 rather than 4 frames per clip (ablation) | TFCU · Celeb-DF, DFDC, and FFIW | frame-wise AUC (%): 91.38 -> 90.55; 85.43 -> 84.80; 90.21 -> 90.13 | Hurt |
Surprising findings
- TFCU underperforms SBI and AltFreezing in video-wise AUC on some face-swapping manipulations despite better frame-wise AUC, attributed to large inter-frame prediction differences in those methods.
- Simply scaling ViT-L to coarse spatiotemporal fusion gives limited gains relative to its parameter increase compared with the TFCU design.
Limitations
- Gains on some face-swapping datasets are reported in frame-wise AUC while video-wise AUC lags behind SBI and AltFreezing.
- Historical review is a post-processing step applied only at inference, so it is not learned end-to-end.
Experimental setup
Datasets: FaceForensics++ c23 DF40
Benchmarks: Celeb-DF DFDC FFIW DF40 cross-manipulation (SadTalker, FOMM, FaceDancer, MobileSwap, SimSwap, InSwapper, UniFace)
Metrics: video-wise AUC frame-wise AUC Params FLOPs FPS
Baselines: RECCE SBI D-adv UIA-ViT CADDM CFM LSDA FFDBackbone FTCN AltFreezing TALL NACO
Builds on: SBI FFDBackbone AltFreezing FTCN SAM 2
Closest papers by meaning
- Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning (CVPR 2025 · similarity 0.88)
- Identity-Aware Vision-Language Model for Explainable Face Forgery Detection (AAAI 2026 · similarity 0.87)
- Stacking Brick by Brick: Aligned Feature Isolation for Incremental Face Forgery Detection (CVPR 2025 · similarity 0.85)
- Learning Discriminative Noise Guidance for Image Forgery Detection and Localization (AAAI 2024 · similarity 0.85)
- On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection (NeurIPS 2024 · similarity 0.85)
- DevFD : Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspaces (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.