Research map / Label-efficient and robust learning / Recognition and long-tail
SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SLIM, a two-stage audio deepfake detector that first learns style-linguistics dependency features from real speech only via self-contrastive loss on frozen Wav2vec-XLSR layers, then trains a light classifier on these plus pretrained embeddings, improving out-of-domain EER.
Problem
Audio deepfake detectors generalize poorly from in-domain (ASVspoof2019) to out-of-domain data (In-the-wild, MLAAD-EN) and their decisions are hard to explain.
Approach
Model the style-linguistics dependency in real speech using a one-class self-supervised contrastive objective on frozen SSL layers (style from SER-finetuned Wav2vec-XLSR, linguistics from ASR-finetuned Wav2vec-XLSR), then fuse these dependency features with the original embeddings for supervised real/fake classification; the dependency features make deepfake style-content mismatch quantifiable.
Key results
- With frozen frontends, SLIM reaches 12.9% EER on In-the-wild and 13.5% on MLAAD-EN, versus 24.2% and 26.5% for the best frozen benchmark (SSL-Fusion).
- On ASVspoof2019 and ASVspoof2021, frozen SLIM reports 0.6% and 8.3% EER, competitive with W2V-LCNN (8.1% on ASVspoof2021) and SSL-Fusion.
- Finetuned SLIM achieves 12.5% EER on In-the-wild and 10.7% on MLAAD-EN, outperforming W2V-AASIST (17.5%, 14.5%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add style-linguistics mismatch dependency features and fuse them with the original representations | SLIM · In-the-wild and MLAAD-EN | EER: 12.9 vs 24.2; 13.5 vs 26.5 | Helped |
| Add style-linguistics mismatch dependency features and fuse them with the original representations | SLIM · ASVspoof2021 DF eval | EER: 8.3 vs 8.1 | Mixed |
| Finetune all modules during Stage 2 (ablation) | SLIM · In-the-wild and MLAAD-EN | EER: 12.9 -> 12.5; 13.5 -> 10.7 | Helped |
| Finetune all modules during Stage 2 (ablation) | SLIM · ASVspoof2019 and ASVspoof2021 | EER: 0.6 -> 0.2; 8.3 -> 4.4 | Helped |
| Use only style embedding at Stage 2 instead of full SLIM (ablation) | SLIM · ASVspoof2019 eval | EER: 0.6 -> 6.7 | Hurt |
| Use only linguistic embedding at Stage 2 instead of full SLIM (ablation) | SLIM · ASVspoof2019 eval | EER: 0.6 -> 5.9 | Hurt |
| Use concatenated style and linguistic embeddings without dependency features (ablation) | SLIM · MLAAD-EN | EER: 13.5 -> 23.9 | Hurt |
| Use only dependency features instead of the full feature fusion (ablation) | SLIM · In-the-wild | EER: 12.9 -> 25.8 | Hurt |
| Remove ASP and MLP backend layers (ablation) | SLIM · ASVspoof2019, ASVspoof2021, In-the-wild, MLAAD-EN | EER: 0.6 -> 0.9; 8.3 -> 9.1; 12.9 -> 13.1; 13.5 -> 13.7 | Hurt |
| Replace ASP+MLP with LLGF backend (ablation) | SLIM · ASVspoof2019 and ASVspoof2021 | EER: 0.6 -> 0.4; 8.3 -> 7.5 | Helped |
| Replace ASP+MLP with LLGF backend (ablation) | SLIM · In-the-wild and MLAAD-EN | EER: 12.9 -> 13.5; 13.5 -> 13.0 | Mixed |
| Replace ASP+MLP with LCNN backend (ablation) | SLIM · ASVspoof2019 and ASVspoof2021 | EER: 0.6 -> 0.3; 8.3 -> 7.9 | Helped |
Surprising findings
- Simply concatenating style and linguistics embeddings gave no significant gain over benchmarks, suggesting the dependency is not captured by supervised training alone.
- Dependency features alone generalize best to out-of-domain data (19.8% EER on MLAAD-EN) but are weak on ASVspoof2019 (2.8%) and ASVspoof2021 (20.5%).
Limitations
- Real speech with atypical style-linguistics dependency (e.g., dysarthric speech) may be misclassified as fake.
- Finetuning the frontend improves performance but compromises the disentangled interpretability of the features.
- Performance on in-domain ASVspoof2021 is only comparable to top baselines and the In-the-wild EER remains around 13%.
Experimental setup
Datasets: Common Voice RAVDESS ASVspoof2019 LA ASVspoof2021 DF In-the-wild MLAAD v3 (English subset)
Benchmarks: ASVspoof2019 LA ASVspoof2021 DF In-the-wild MLAAD-EN
Metrics: EER F1
Baselines: W2V-LLGF W2V-LCNN SSL-Fusion LCNN RawNet2 PS3DT W2V-ASP-ft W2V-AASIST
Builds on: CCA Barlow Twins-style redundancy reduction XLSR
Closest papers by meaning
- I Can Hear You: Selective Robust Training for Deepfake Audio Detection (ICLR 2025 · similarity 0.86)
- Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection (NeurIPS 2026 · similarity 0.85)
- Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes (AAAI 2025 · similarity 0.85)
- Circumventing Shortcuts in Audio-visual Deepfake Detection Datasets with Unsupervised Learning (CVPR 2025 · similarity 0.83)
- Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection (AAAI 2025 · similarity 0.83)
- Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.