Research map / Label-efficient and robust learning / Self-supervised and contrastive
SPEAR: A Unified SSL Framework for Learning Speech and Audio Representations
Xiaoyu Yang, Yifan Yang, Zengrui Jin, Ziyun Cui, Wen Wu, Baoxiang Li, Chao Zhang, Phil Woodland
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add token mixing augmentation (ablation) | SPEAR-Speech Base · SUPERB SD | DER: 4.23 -> 3.01 | Improved |
| add token mixing augmentation (ablation) | SPEAR-Speech Base · SUPERB SS | SI-SDRi: 10.52 -> 11.17 | Improved |
| add token mixing augmentation (ablation) | SPEAR-Speech Base · LibriSpeech LS-100 | WER: 3.1 -> 3.1 (clean) | No clear effect |
| add token mixing augmentation (ablation) | SPEAR-Speech Base · SUPERB PR | PER: 3.3 -> 3.4 | No clear effect |
| use Asymmetrical instead of Disjoint dual-domain loss (ablation) | SPEAR-SA Large · HEAR | average score: 78.3 -> 79.0 | Improved |
| use Asymmetrical instead of Joint dual-domain loss (ablation) | SPEAR-SA Large · HEAR | average score: 78.7 -> 79.0 | Improved |
| pre-train on speech and audio jointly instead of speech only | SPEAR Large · HEAR | average score: 72.96 -> 79.26 | Improved |
| pre-train on speech and audio jointly instead of speech only | SPEAR Large · LibriSpeech LS-960 test-other | WER: 3.3 -> 3.4 | Hurt |
| use MVQ tokens instead of k-means targets (ablation) | SPEAR · SUPERB SID, ER | accuracy | Improved |
| replace Zipformer encoder with standard Transformer (ablation) | SPEAR · ASR fine-tuning | efficiency and WER | Hurt |
| increase number of MVQ codebooks (ablation) | SPEAR · speech vs general audio | downstream performance | Mixed |
| scale to XLarge with 197k hours | SPEAR-SA · AudioSet AS-2M | mAP: 49.6 -> 50.0 | Improved |
Closest papers by meaning
- CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing (NeurIPS 2024 · similarity 0.88)
- SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes (ICLR 2025 · similarity 0.87)
- MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training (ICLR 2024 · similarity 0.86)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.86)
- SelfVC: Voice Conversion With Iterative Refinement using Self Transformations (ICML 2024 · similarity 0.85)
- MATS: An Audio Language Model under Text-only Supervision (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
