Research map / Label-efficient and robust learning / Self-supervised and contrastive
ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add speaker disentanglement (ContentVec full framework) vs HuBERT-iter | HuBERT-based SSL · Zero-resource content probe | ABX(w) ↓: 6.01 -> 5.13 | Helped |
| Remove voice-conversion teacher disentanglement (No-DTeachers) (ablation) | ContentVec · Ablation Librispeech | ABX(w) ↓: 5.13 -> 5.67 | Hurt |
| Remove student transformations and contrastive loss (No-DStudents) (ablation) | ContentVec · Ablation Librispeech | ABX(w) ↓: 5.13 -> 5.62 | Hurt |
| Remove speaker conditioning of predictor (No-Cond) (ablation) | ContentVec · Ablation Librispeech | ABX(w) ↓: 5.13 -> 5.76 | Hurt |
| Remove speaker conditioning of predictor (No-Cond) (ablation) | ContentVec · Ablation Librispeech | PNMI ↑: 0.590 -> 0.593 | Mixed |
| Move contrastive loss to last layer (N1) (ablation) | ContentVec · Position of contrastive loss | ABX(w) ↓: 5.13 -> 5.76 | Hurt |
| Contrastive loss weight 5e-5 vs standard 1e-5 (ablation) | ContentVec · Contrastive loss weight | ABX(w) ↓: 5.13 -> 5.22 | No clear effect |
| Use HuBERT-iter teacher without voice conversion instead of ContentVec (iterative training only) | HuBERT · SUPERB content/semantic | QbE MTWV ↑: 0.0891 (HuBERT-iter) vs 0.0590 (ContentVec) | Mixed |
| Apply speaker disentanglement (ContentVec) vs HuBERT in language modeling | discrete unit LM · Language modeling PPX | PPX ↓: 790.17 -> 650.04 | Helped |
| Use ContentVec continuous features in voice conversion vs HuBERT | voice conversion · Librispeech C2C | d-vector cosine similarity ↑: 0.9029 -> 0.9316 | Helped |
| Reduce voice converter training steps for teacher (10k instead of 40k) (ablation) | ContentVec · Voice converter quality ablation | ABX(w) ↓: 5.13 -> 6.05 | Hurt |
| Impose contrastive loss on multiple layers (last, last-but-three, last-but-seven) (ablation) | ContentVec · Multi-layer contrastive loss | ABX(w) ↓: 5.13 -> 5.19 | Hurt |
Closest papers by meaning
- Self-Supervised Disentangled Representation Learning for Robust Target Speech Extraction (AAAI 2024 · similarity 0.91)
- SelfVC: Voice Conversion With Iterative Refinement using Self Transformations (ICML 2024 · similarity 0.87)
- CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing (NeurIPS 2024 · similarity 0.86)
- Speech Self-Supervised Learning Using Diffusion Model Synthetic Data (ICML 2024 · similarity 0.86)
- Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations (AAAI 2024 · similarity 0.85)
- Let There Be Sound: Reconstructing High Quality Speech from Silent Videos (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.