Research map / Vision-language and multimodal / Cross-modal learning
Joint Audio-Visual Deepfake Detection
Yipin Zhou, Ser-Nam Lim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| late-fuse video and audio streams (ablation) | Indp · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 95.83 -> 94.25 | Hurt |
| add sync-stream with central connections at low and high layers (no attention) (ablation) | Late-fuse · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 94.25 -> 97.02 | Improved |
| add sync-stream (central connections) (ablation) | Indp · DFDC (audiovisual) | whole-video accuracy (%): 89.36 -> 91.01 | Improved |
| use inter-attention instead of inter+intra attention (ablation) | {2+1}-streams · FF (FaceForensics++ audiovisual) | audio-stream accuracy (%): 94.25 -> 96.03 | Improved |
| use inter-attention instead of joint attention (ablation) | {2+1}-streams · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 96.23 -> 97.62 | Improved |
| add inter-attention to sync-stream (ablation) | {2+1}-streams · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 97.02 -> 97.62 | Improved |
| add attention to sync-stream (ablation) | {2+1}-streams · DFDC (audiovisual) | whole-video accuracy (%): 91.01 -> 90.84 | No clear effect |
| freeze independently trained video and audio streams (Fix-indp-audvid) (ablation) | {2+1}-streams Inter · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 97.62 -> 96.63 | Hurt |
| freeze independently trained streams in late-fuse model (ablation) | Late-fuse · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 94.25 -> 94.84 | Improved |
| connect audio and visual features at more lower layers (cnt-1234 vs late-fuse) (ablation) | {2+1}-stream · FF (FaceForensics++ audiovisual) | whole-video accuracy (%) | Improved |
| train with shuffled (unpaired) audio (ablation) | {2+1}-streams+att · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 97.62 -> 95.44 | Hurt |
| train with shuffled audio versus independent streams | Indp · FF (FaceForensics++ audiovisual) | whole-video accuracy (%): 95.83 -> 95.44 | No clear effect |
Closest papers by meaning
- AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection (CVPR 2024 · similarity 0.91)
- Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning (AAAI 2025 · similarity 0.90)
- Multi-View Collaborative Learning Network for Speech Deepfake Detection (AAAI 2025 · similarity 0.89)
- Circumventing Shortcuts in Audio-visual Deepfake Detection Datasets with Unsupervised Learning (CVPR 2025 · similarity 0.88)
- I Can Hear You: Selective Robust Training for Deepfake Audio Detection (ICLR 2025 · similarity 0.88)
- Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
