Research map / Video / Video generation
GLCF: A Global-Local Multimodal Coherence Analysis Framework for Talking Face Generation Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MSTF, a large multi-scenario talking-face deepfake dataset (22 forgery techniques, ~144k audio-visual videos), and GLCF, a global-local audiovisual coherence detector that outperforms prior deepfake detectors on MSTF and other benchmarks.
Problem
Detecting talking face generation (TFG) videos is hard because existing deepfake detectors transfer poorly to high-quality, audio-synchronized talking-face content, and public datasets for TFG are lacking.
Approach
Combines a global temporal branch (region-focused smoothness module on inter-frame differences plus a discrepancy-capture attention module with variance-based activation) with a local visual-audio cross-attention fusion module that checks audiovisual coherence, alongside a frequency-statistics branch.
Key results
- On MSTF, GLCF reaches 0.8849 accuracy, beating the second-best MiNet (0.8612) by 2.37 points.
- On FakeAVCeleb, GLCF reaches 0.9892 accuracy, versus 0.9833 for Multi-att.
- Ablation on MSTF: removing V-AFM drops accuracy from 0.8849 to 0.8022, the largest drop among the modules.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GLCF instead of the strongest listed baseline | Talking face detection · MSTF (intra-dataset) | accuracy: 0.8612 -> 0.8849 | Helped |
| Use GLCF instead of the strongest listed baseline | Multimodal forgery detection · FakeAVCeleb | accuracy: 0.9833 -> 0.9892 | Helped |
| Use GLCF instead of the strongest listed baseline | Deepfake detection · DFDC | accuracy: 0.9245 -> 0.9258 | Helped |
| Use GLCF instead of the strongest listed baseline | Forgery detection trained on FakeAVCeleb · MSTF (train FakeAVCeleb) | accuracy: 0.6000 -> 0.6194 | Helped |
| add LFS (ablation) | GLCF · MSTF | accuracy: 0.8625 -> 0.8849 | Helped |
| add RFSDM (ablation) | GLCF · MSTF | accuracy: 0.8503 -> 0.8849 | Helped |
| add DCTAM (ablation) | GLCF · MSTF | accuracy: 0.8470 -> 0.8849 | Helped |
| add V-AFM (ablation) | GLCF · MSTF | accuracy: 0.8022 -> 0.8849 | Helped |
Surprising findings
- MSTF is reported as harder to detect than FF++ (C40) for the compared methods.
- Methods trained on FakeAVCeleb generalize poorly to MSTF, with several scoring near chance (e.g., Joint A-V 0.4124).
Limitations
- Does not address detection of TFG videos compressed by social media platforms.
- Some baselines (RECCE, Joint A-V, AVD2-DWF) were not fully open-sourced and were reimplemented by the authors.
Experimental setup
Datasets: MSTF FaceForensics++ FakeAVCeleb DFDC CelebA-HQ CREMA-D VoxCeleb VoxCeleb2 MEAD LRS2 LibriSpeech ASVspoof2019
Benchmarks: MSTF FaceForensics++ (C40) FakeAVCeleb DFDC
Metrics: accuracy
Baselines: F3Net RFM Multi-att RECCE MiNet Joint A-V AVSSD AVD2-DWF
Builds on: Local Frequency Statistic Method Grad-CAM
Closest papers by meaning
- TalkCuts: A Large-Scale Dataset for Multi-Shot Human Speech Video Generation (NeurIPS 2025 · similarity 0.85)
- Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation (CVPR 2025 · similarity 0.85)
- From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speech (CVPR 2025 · similarity 0.85)
- Faces that Speak: Jointly Synthesising Talking Face and Speech from Text (CVPR 2024 · similarity 0.85)
- Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model (ICML 2025 · similarity 0.85)
- Learning Adaptive Spatial Coherent Correlations for Speech-Preserving Facial Expression Manipulation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.