Research map / Adversarial robustness and security / Certified and empirical robustness
I Can Hear You: Selective Robust Training for Deepfake Audio Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds DeepFakeVox-HQ, a 1.3M-sample deepfake voice dataset, and proposes F-SAT, a frequency-selective adversarial training method with RandAugment-style audio augmentation to make RawNet3 detectors robust to corruptions and attacks.
Problem
Deepfake audio detectors reported as highly accurate on public benchmarks generalize poorly to diverse, newer deepfakes and fail under realistic corruptions and adversarial attacks, and they rely on high-frequency features that are easy to manipulate.
Approach
Adversarial training that perturbs only the STFT magnitude within a chosen high-frequency band (e.g., 4-8 kHz) so the detector becomes robust to high-frequency manipulation without degrading low-frequency cues or clean accuracy, combined with a diversified audio augmentation policy and a large new training corpus.
Key results
- F-SAT with RandAug on RawNet3 improves average accuracy on DeepFakeVox-HQ test by 7.7 points over RawNet3 (98.0% vs 90.3%) and reaches 96.6% on ASVspoof2019.
- On frequency-domain attacks (0-8kHz), RawNet3 fake accuracy rises from 17.9% to 92.8% with RandAug+F-SAT in the ablation table.
- Adversarial training on the time domain reduces clean accuracy and does not improve robustness, while magnitude-based 4-8kHz F-SAT gives the best overall trade-off.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with DeepFakeVox-HQ using RandAug and F-SAT instead of RawNet3 baseline | RawNet3 · DeepFakeVox-HQ test | Avg accuracy: 90.3% -> 98.0% | Helped |
| Train with DeepFakeVox-HQ using RandAug and F-SAT instead of RawNet3 baseline | RawNet3 · ASVspoof2019 | Avg accuracy: 93.9% -> 96.6% | Helped |
| Train with DeepFakeVox-HQ using RandAug and F-SAT instead of RawNet3 baseline | RawNet3 · WaveFake | Avg accuracy: 99.8% -> 99.9% | Helped |
| add RandAug (ablation) | RawNet3 · DeepFakeVox-HQ test, corruption | Fake accuracy: 55.6% -> 85.6% | Helped |
| add RandAug (ablation) | RawNet3 · DeepFakeVox-HQ test, original | Fake accuracy: 84.4% -> 97.0% | Helped |
| add F-SAT to RandAug training (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, Attack (Time) | Fake accuracy: 66.0% -> 87.0% | Helped |
| add F-SAT to RandAug training (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, Attack (Frequency) | Fake accuracy: 62.4% -> 92.8% | Helped |
| use time-domain adversarial training instead of F-SAT (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, Attack (Time) | Fake accuracy: 66.0% -> 12.4% | Hurt |
| use time-domain adversarial training instead of F-SAT (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, original | Fake accuracy: 97.0% -> 83.1% | Hurt |
| target 4-8kHz rather than 0-8kHz in F-SAT (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, original | Fake accuracy: 0.892 -> 0.984 | Helped |
| target 4-8kHz rather than 0-8kHz in F-SAT (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, Attack (4-8kHz) | Fake accuracy: 0.848 -> 0.970 | Helped |
| target 6-8kHz rather than 4-8kHz in F-SAT (ablation) | RawNet3+RandAug · DeepFakeVox-HQ test, original | Fake accuracy: 0.984 -> 0.974 | Hurt |
Surprising findings
- Time-domain adversarial training diminished accuracy on unattacked data and failed to improve robustness against attacks in either domain.
- Spectrum-based TE-ResNet benefits from F-SAT on the 0-4kHz band, the opposite of the high-frequency focus found for RawNet3.
Limitations
- Real-world test fake samples from social media and commercial tools are small (about 7,000 total test samples), limiting statistical strength.
- Adversarial robustness evaluated mainly with gradient-based attacks at a fixed epsilon; generalization to unseen attacks is limited by the authors' own account of adversarial training.
Experimental setup
Datasets: DeepFakeVox-HQ (train) VCTK LibriSpeech In-The-Wild AudioSet ASVspoof2019 ASVspoof2021 VoxCeleb1 WaveFake
Benchmarks: DeepFakeVox-HQ (test) ASVspoof2019 WaveFake
Metrics: accuracy on real accuracy on fake average accuracy
Baselines: RawNet3 RawNet2 RawGAT-ST TE-ResNet RandAugment-only RawNet3 time-domain adversarial training
Builds on: RawNet3 RandAugment PGD-style iterative adversarial training Sinc-layers
Closest papers by meaning
- Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection (NeurIPS 2026 · similarity 0.89)
- Circumventing Shortcuts in Audio-visual Deepfake Detection Datasets with Unsupervised Learning (CVPR 2025 · similarity 0.88)
- Multi-View Collaborative Learning Network for Speech Deepfake Detection (AAAI 2025 · similarity 0.87)
- Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes (AAAI 2025 · similarity 0.87)
- Region-Based Optimization in Continual Learning for Audio Deepfake Detection (AAAI 2025 · similarity 0.86)
- AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.