Research map / Video / Video generation
Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Content
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UNITE, a SigLIP-feature transformer trained with cross-entropy plus an attention-diversity loss to detect face, background, and fully synthetic (T2V/I2V) video manipulations.
Problem
Face-centric DeepFake detectors fail on background manipulations and fully AI-generated text-to-video and image-to-video content, and existing detectors need faces to run.
Approach
A frozen SigLIP-So400M encoder provides per-frame domain-agnostic tokens to a 4-block transformer, and an attention-diversity loss (Fisher-style within/between-class terms on attention-pooled features) pushes attention heads away from the face so the model covers full frames. Training also mixes in the fully synthetic GTA-V (SAIL-VOS-3D) data.
Key results
- Trained on FF++ with GTA-V, UNITE reaches 100% accuracy on AVID and GTA-V and 87.12% on DeMamba, versus 41.67% and 61.47% when trained on FF++ alone.
- On face-manipulated benchmarks UNITE reports 95.11% on CelebDF and 99.62% on DeeperForensics, outperforming the listed face-specific baselines in Table 2.
- On the DeMamba validation split UNITE reports 89.60% average recall and 92.76% average precision in Table 3, without training on DeMamba, versus the reported baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GTA-V training data to FF++ training | UNITE · Face-manipulated datasets (FF++, CelebDF, DeeperForensics, DeepfakeTIMIT, HifiFace, UADFV) | accuracy: 72.61% -> 95.11% on CelebDF; 63.63% -> 75.62% on HifiFace | Helped |
| add GTA-V training data to FF++ training | UNITE · AVID | accuracy: 41.67% -> 100.00% | Helped |
| add GTA-V training data to FF++ training | UNITE · GTA-V and DeMamba | accuracy: 60.16% -> 100.00% on GTA-V; 61.47% -> 87.12% on DeMamba | Helped |
| add GTA-V training data to FF++ training | UNITE · NYTimes | accuracy: 50.00% -> 80.00% | Helped |
| combine cross-entropy loss with attention-diversity loss (ablation) | UNITE · FF++, CelebDF, UADFV, AVID, GTA-V, DeMamba | accuracy | Helped |
| add attention-diversity loss (ablation) | UNITE trained on FF++ · AVID | accuracy | Helped |
| add attention-diversity loss (ablation) | UNITE trained on FF++ and GTA-V · DeMamba | accuracy | Helped |
| use only cross-entropy loss instead of cross-entropy plus attention-diversity loss (ablation) | UNITE · DeMamba | accuracy | Hurt |
| increase sampled frames per video segment from 1 to 64 (ablation) | UNITE · FF++, UADFV, AVID, GTA-V, DeMamba | accuracy | Helped |
| use transformer depth 4 rather than depths 2, 6, or 8 (ablation) | UNITE · FF++, UADFV, AVID, GTA-V, DeMamba | accuracy | Mixed |
| add GTA-V training data to FF++ training (ablation) | UNITE fine-grained classifier · FF++, CelebDF, DeeperForensics, DeepfakeTIMIT, HifiFace, UADFV, AVID, GTA-V, DeMamba | accuracy: 96.46% -> 97.70% on FF++; 60.40% -> 70.17% on CelebDF; 0.00% -> 100.00% on GTA-V | Helped |
| train without fully synthetic GTA-V data (ablation) | UNITE fine-grained classifier · GTA-V | accuracy: 0.00% -> 100.00% | Hurt |
Surprising findings
- Adding GTA-V synthetic game footage, which is not AI-generated, substantially improves detection of AI-generated T2V/I2V videos and background edits.
- CE-only training on FF++ gives 0% accuracy on the AVID background-manipulation set, showing face-focused attention fails completely on background edits.
Limitations
- Relies on a single 8-TPU training setup and a small set of evaluation sets; in-the-wild evaluation uses only 10 NYTimes quiz videos.
- Synthetic GTA-V training data is not AI-generated, and the paper notes cross-dataset comparisons against DeMamba are not strictly fair because UNITE was not trained on DeMamba.
- Uses frame sampling every other frame and padding for short videos; detection of unseen generators is assessed only on a few datasets.
Experimental setup
Datasets: FaceForensics++ SAIL-VOS-3D GTA-V
Benchmarks: FaceForensics++ CelebDF DeeperForensics DeepfakeTIMIT HifiFace UADFV AVID GTA-V DeMamba NYTimes DeepFake quiz
Metrics: accuracy AUC precision recall precision at 0.8 recall recall at 0.8 precision
Baselines: TALL ISTVT TI2Net LRNet DeMamba NPR STIL F3Net MINTIME FTCN XCLIP
Builds on: SigLIP Fisher Discriminant Analysis TALL DeMamba
Closest papers by meaning
- Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning (CVPR 2025 · similarity 0.88)
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.88)
- Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection (AAAI 2025 · similarity 0.87)
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.87)
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.87)
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.