Research map / Label-efficient and robust learning / Recognition and long-tail
Multi-View Collaborative Learning Network for Speech Deepfake Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a dual-branch (waveform Transformer + spectrogram CNN) network with contrastive learning and cross-attention fusion modules that jointly learns from raw waveforms and log spectrograms for speech deepfake detection.
Problem
Speech deepfake detection methods use either raw waveform or spectrogram features alone, ignoring complementary correlations between the two views, which hurts generalization to unseen synthesizers.
Approach
Jointly train a waveform branch and a spectrogram branch with inner- and inter-view contrastive losses and a Waveform-Spectrogram Fusion Module that exchanges and cross-attends features between views at each stage, rather than combining separately pretrained multi-view features.
Key results
- On ASVspoof2021 DF, the method reaches 7.77% EER in ablation and outperforms all baselines across attack categories; on the ASVspoof2021 LA test it achieves 1.56% EER (AUC 99.64).
- On MLAAD cross-language evaluation (trained on EN, ES, DE), the method achieves 4.73% EER on the full test set versus higher errors for baselines such as AASIST and MPE.
- Robustness: under 20 dB background noise on ASVspoof2021 DF the method obtains 11.58% average EER, and on compression artifacts (trained on 2019 LA) it reaches 1.22% EER on 2019 LA and 10.20% on 2021 DF AR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-view collaborative learning from waveform and spectrogram features | speech deepfake detection · ASVspoof2021 DF testing subset | EER: 20.01 -> 9.33 | Helped |
| add inner-view contrastive learning (ablation) | Multi-View Collaborative Learning Network · ASVspoof2021 DF | EER: 12.71% vs. 7.77% | Helped |
| add inter-view contrastive learning (ablation) | Multi-View Collaborative Learning Network · ASVspoof2021 DF | EER: 13.52% vs. 7.77% | Helped |
| add WSFM feature interaction (ablation) | Multi-View Collaborative Learning Network · ASVspoof2021 DF | EER: 10.47% vs. 7.77% | Helped |
| add single-view classification losses (ablation) | Multi-View Collaborative Learning Network · ASVspoof2021 DF | EER: 12.21% vs. 7.77% | Helped |
| use multi-view collaborative learning from waveform and spectrogram features | speech deepfake detection · MLAAD Full | EER: 9.42 -> 4.73 | Helped |
Limitations
- Cross-attention fusion has quadratic complexity, so longer sequences increase training cost.
- Evaluated on speech datasets only; generalization beyond the tested synthesizers and languages is not established.
Experimental setup
Datasets: ASVspoof2019 LA ASVspoof2021 LA ASVspoof2021 DF MLAAD MUSAN
Benchmarks: ASVspoof2021 LA ASVspoof2021 DF MLAAD ASVspoof2019 LA
Metrics: AUC EER
Baselines: LCNN RawNet2 AASIST LibriSeVoc Wav2Clip AudioClip ABC-CapsNet MPE ASDG RawGAT
Builds on: ResNet Transformer (wav2vec-style 12-layer Transformer) AASIST RawNet2 supervised contrastive learning
Closest papers by meaning
- Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes (AAAI 2025 · similarity 0.88)
- Multi-modal Deepfake Detection via Multi-task Audio-Visual Prompt Learning (AAAI 2025 · similarity 0.87)
- AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection (CVPR 2024 · similarity 0.87)
- I Can Hear You: Selective Robust Training for Deepfake Audio Detection (ICLR 2025 · similarity 0.87)
- Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection (AAAI 2025 · similarity 0.85)
- Region-Based Optimization in Continual Learning for Audio Deepfake Detection (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.