Research map / Label-efficient and robust learning / Self-supervised and contrastive
DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning
Alexander H. Liu, Heng-Jui Chang, Michael Auli, Wei-Ning Hsu, James R. Glass
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| online clustering with codebook instead of continuous self-distillation (data2vec) | DinoSR · ZeroSpeech 2021 ABX average | ABX error rate (%): 5.20 -> 3.59 | Improved |
| online clustering instead of offline iterative clustering (HuBERT) | DinoSR · ZeroSpeech 2021 ABX average | ABX error rate (%): 4.22 -> 3.59 | Improved |
| DinoSR vs data2vec fine-tuning | DinoSR · LibriSpeech test-other, 10 min | WER (%): 12.3 -> 11.8 | Improved |
| DinoSR vs data2vec fine-tuning | DinoSR · LibriSpeech test-other, 100 hr | WER (%): 6.8 -> 6.7 | No clear effect |
| DinoSR vs data2vec fine-tuning | DinoSR · LibriSpeech dev-clean, 100 hr | WER (%): 2.2 -> 2.3 | Hurt |
| DinoSR vs data2vec on SUPERB PR | DinoSR · SUPERB PR | PER (%): 4.69 -> 3.21 | Improved |
| DinoSR vs WavLM on SUPERB slot filling CER | DinoSR · SUPERB SF | CER (%): 22.86 -> 23.57 | Hurt |
| vary codebook size V from 64 to 2048 (ablation) | DinoSR · LibriSpeech dev-other, 10 hr fine-tune | WER | No clear effect |
| vary the top N layers that apply online clustering (ablation) | DinoSR · LibriSpeech dev-other, 10 hr fine-tune | WER | Mixed |
| vary codebook decay rate tau between 0.5 and 0.99 (ablation) | DinoSR · LibriSpeech dev-other, 10 hr fine-tune | WER | No clear effect |
| anneal codebook decay tau to 0.99 or 0.999 (ablation) | DinoSR · LibriSpeech dev-other, 10 hr fine-tune | WER | No clear effect |
| DinoSR online codebook vs VQ-APC code usage | DinoSR · LibriSpeech dev unit quality | code perplexity: 72.1 -> 179.2 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- ContentVec — ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers (ICML 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Self-Supervised Disentangled Representation Learning for Robust Target Speech Extraction (AAAI 2024 · similarity 0.85)
- SelfVC: Voice Conversion With Iterative Refinement using Self Transformations (ICML 2024 · similarity 0.85)
- Clustering Properties of Self-Supervised Learning (ICML 2025 · similarity 0.85)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.84)
- SECodec: Structural Entropy-based Compressive Speech Representation Codec for Speech Language Models (AAAI 2025 · similarity 0.84)
- Sylber: Syllabic Embedding Representation of Speech from Raw Audio (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
