Research map / Label-efficient and robust learning / Recognition and long-tail

Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes

AAAI 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Multilingual language representation pretraining · 12 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes phoneme-level deepfake speech detection using adaptive phoneme pooling over WavLM frame features, a graph attention network over phoneme sequences, and random phoneme substitution augmentation, reporting state-of-the-art EER across ASVspoof2019/2021, MLAAD, and InTheWild.

Problem

Detecting speech deepfakes that generalize to unseen synthesizers, languages, and datasets; prior phoneme-based detectors focus on specific phonemes and ignore temporal dynamics across the whole phoneme sequence.

Approach

Average consecutive frames sharing the same predicted phoneme label into sample-specific phoneme-level features, model transitions between adjacent phonemes with a GAT plus LSTM, and augment training by swapping same-phoneme segments across samples, with a CLIP-style loss aligning frame features.

Key results

• On ASVspoof2021 DF whole testing set, the method achieves 98.39% AUC and 7.12% EER, versus 93.04% AUC / 14.81% EER for the best prior method RawGAT.
• On InTheWild, the method reaches 16.07% EER (91.52% AUC) when trained on MLAAD EN/DE/ES subsets, compared with 26.40%-86.85% for baselines.
• Under compression artifacts (trained on ASVspoof2019 LA), the whole-set EER is 10.42% versus 19.02% for the best baseline AASIST.

Measured effects reported in the paper

Surprising findings

• Phoneme-level features from WavLM separate real and fake samples more clearly in t-SNE than Wav2Vec2 features, suggesting WavLM generalizes better as a backbone.

Limitations

• Phoneme recognizer is weak (PER about 0.4 on validation) due to compute constraints, which may limit the phoneme-level features.
• Relies on predicted phoneme labels and requires a multilingual phoneme recognizer trained on Common Voice.
• Ablation is reported only on InTheWild and the CLIP-loss contribution is inconsistently described across ablation rows.

Experimental setup

Datasets: Common Voice 6.1 ASVspoof2019 LA ASVspoof2021 DF MLAAD InTheWild MUSAN

Benchmarks: ASVspoof2021 DF MLAAD InTheWild ASVspoof2019 LA

Metrics: AUC EER

Baselines: LCNN RawNet2 RawGAT AASIST LibriSeVoc Wav2Clip AudioClip ABC-CapsNet MPE ASDG

Builds on: Wav2Vec2 WavLM graph attention network CLIP CTC

Closest papers by meaning

• Multi-View Collaborative Learning Network for Speech Deepfake Detection (AAAI 2025 · similarity 0.88)
• I Can Hear You: Selective Robust Training for Deepfake Audio Detection (ICLR 2025 · similarity 0.87)
• Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection (NeurIPS 2026 · similarity 0.85)
• SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detection (NeurIPS 2024 · similarity 0.85)
• Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning (AAAI 2025 · similarity 0.85)
• Circumventing Shortcuts in Audio-visual Deepfake Detection Datasets with Unsupervised Learning (CVPR 2025 · similarity 0.84)