Research map / Label-efficient and robust learning / Domain adaptation and generalization
Improving Generalization for AI-Synthesized Voice Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a disentanglement framework that separates domain-agnostic vocoder artifact features from domain-specific ones and trains them with sharpness-aware minimization to improve cross-vocoder generalization of AI-synthesized voice detection.
Problem
AI-synthesized voice detectors excel on seen vocoders but generalize poorly to unseen vocoders and are sensitive to background noise and speaker identity.
Approach
Disentangle content, domain-specific artifact, and domain-agnostic artifact features using classification, contrastive, reconstruction and mutual-information losses, then train with SAM to flatten the loss landscape; the domain-agnostic head is used for inference.
Key results
- Trained on LibriSeVoc, reaches 13.55% average EER on seen vocoders vs 18.67% for Sun et al. (about 5.12% improvement).
- Cross-domain average EER on unseen vocoders is 20.27% vs 27.86% for Sun et al. when trained on LibriSeVoc (about 7.59% improvement).
- On the WaveFake unseen subset (trained on LibriSeVoc), average EER is 23.38% vs 34.06% for the best prior baseline (Sun et al.).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the full disentanglement framework | RawNet2 · LibriSeVoc seen vocoders | EER (%): 18.67% vs. 13.55% | Helped |
| use the full disentanglement framework | RawNet2 · LibriSeVoc unseen vocoders | EER (%): 27.86% vs. 20.27% | Helped |
| add reconstruction module (ablation) | RawNet2 · LibriSeVoc seen vocoders | EER (%): 20.21 -> 19.16 | Mixed |
| add reconstruction module (ablation) | RawNet2 · LibriSeVoc unseen vocoders | EER (%): 30.16 -> 28.57 | Helped |
| add multi-task classification head (ablation) | V_A · LibriSeVoc seen vocoders | EER (%): 19.16 -> 16.58 | Helped |
| add multi-task classification head (ablation) | V_A · LibriSeVoc unseen vocoders | EER (%): 28.57 -> 26.07 | Helped |
| add contrastive learning (ablation) | V_B · LibriSeVoc seen vocoders | EER (%): 16.58 -> 15.90 | Helped |
| add contrastive learning (ablation) | V_B · LibriSeVoc unseen vocoders | EER (%): 26.07 -> 26.83 | Hurt |
| add mutual information module (ablation) | V_C · LibriSeVoc unseen vocoders | EER (%): 26.83 -> 23.23 | Helped |
| add SAM (ablation) | V_D · LibriSeVoc seen vocoders | EER (%): 15.71 -> 13.55 | Helped |
| add SAM (ablation) | V_D · LibriSeVoc unseen vocoders | EER (%): 23.23 -> 20.27 | Helped |
| increase mutual-information weight λ4 (ablation) | Ours · seen and unseen data | EER | Mixed |
Surprising findings
- Foundation-model baselines WavLM and XLS-R perform worse than the small RawNet2-based models on these detection benchmarks.
- On ASVspoof2019 the method's cross-domain gain over Sun et al. is small (38.20% vs 39.55%), and LCNN is best on the WF vocoder subset.
Limitations
- Requires training datasets containing synthesized voices from multiple vocoders; single-vocoder datasets limit domain-agnostic feature extraction.
- Gains on some cross-domain subsets are small (e.g., 1.34% on ASVspoof2019 cross-domain vs Sun et al.).
Experimental setup
Datasets: LibriSeVoc ASVspoof2019 WaveFake FakeAVCeleb
Benchmarks: LibriSeVoc ASVspoof2019 WaveFake FakeAVCeleb
Metrics: EER
Baselines: LCNN RawNet2 WavLM XLS-R Sun et al.
Builds on: RawNet2 sharpness-aware minimization Donsker-Varadhan representation AdaIN
Closest papers by meaning
- SelfVC: Voice Conversion With Iterative Refinement using Self Transformations (ICML 2024 · similarity 0.86)
- I Can Hear You: Selective Robust Training for Deepfake Audio Detection (ICLR 2025 · similarity 0.85)
- DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion (AAAI 2024 · similarity 0.85)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.84)
- RVCBench: Benchmarking Robustness of Voice Cloning Across Modern Audio Generation Models (NeurIPS 2026 · similarity 0.84)
- DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.