Research map / Label-efficient and robust learning / Recognition and long-tail
SCOREQ: Speech Quality Assessment with Contrastive Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCOREQ, a batch-all triplet loss adapted to regression (with optional adaptive margin) for training speech quality (MOS) predictors that generalise better to unseen degradation sets and domains than L2-trained models.
Problem
No-reference speech quality metrics trained with L2 loss generalise poorly to out-of-distribution and out-of-domain speech, and L2 training yields fragmented embeddings that do not follow the continuous MOS ordering.
Approach
Adapts the batch-all triplet loss to regression by building on-the-fly validity masks from MOS distances over all triplets in a batch, removing easy triplets, and optionally using an MOS-distance adaptive margin; the encoder is pretrained with this loss and a linear MOS head is then trained on frozen features (NR mode).
Key results
- On NISQA-trained models, NR-SCOREQ reaches PC 0.97 on NISQA TEST FOR and 0.79 on TENCENT-Rev, versus 0.74 for L2-trained model on TENCENT-Rev (Table 2 / Table 3 comparisons).
- Embedding analysis on TCD VOIP: L2 loss embeddings have NMI 0.39 and PC 0.53 while SCOREQ has NMI 0.11 and PC 0.80.
- SCOREQ improves over L2 in most ODS and ODM sets; the w2vlight variant also improves PC on every test set in its appendix table, with SCOREQ Adapt improving over L2 on 10 of 11 sets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace L2 loss with SCOREQ contrastive regression loss | wav2vec 2.0 BASE speech-quality model trained on NISQA TRAIN SIM · VoiceMOS Test 2 | PC: 0.69 -> 0.82 | Helped |
| Replace L2 loss with SCOREQ contrastive regression loss | wav2vec 2.0 BASE speech-quality model trained on NISQA TRAIN SIM · TCD VOIP | PC: 0.80 -> 0.85 | Helped |
| Replace L2 loss with SCOREQ contrastive regression loss | wav2vec 2.0 BASE speech-quality model trained on NISQA TRAIN SIM · P23 EXP3 | PC: 0.95 -> 0.94 | Hurt |
| Replace L2 loss with SCOREQ contrastive regression loss | wav2vec 2.0 BASE speech-quality model trained on VoiceMOS Train · NISQA TEST FOR | PC: 0.82 -> 0.87 | Helped |
| Replace L2 loss with SCOREQ contrastive regression loss | wav2vec 2.0 BASE speech-quality model trained on VoiceMOS Train · VoiceMOS Test 2 | PC: 0.98 -> 0.97 | Hurt |
| Use adaptive rather than fixed margin (ablation) | SCOREQ speech-quality model trained on NISQA TRAIN SIM · NISQA TEST FOR | PC: 0.96 -> 0.97 | No clear effect |
| Use batch-all in-batch triplet mining rather than offline triplet sampling (ablation) | SCOREQ speech-quality model · NISQA TRAIN SIM test sets | PC | Helped |
| Use SCOREQ Adapt rather than L2 loss | w2vlight speech-quality model trained on NISQA TRAIN SIM · VoiceMOS Test 2 | PC: 0.75 -> 0.69 | Hurt |
Surprising findings
- The offline NOMAD-style triplet model underperforms the L2 loss baseline on several sets, suggesting that offline triplet mining hurts.
- The adaptive margin gives only minimal benefit over a constant margin; most gain comes from the batch-all strategy.
Limitations
- Only evaluated on speech domains; not tested on other regression tasks.
- Cross-domain setting between encoder training and MOS mapping was not explored.
- SCOREQ hyperparameters did not receive an equivalent grid search as the L2 baseline.
Experimental setup
Datasets: NISQA TRAIN SIM NISQA VAL SIM VoiceMOS Train DNS Squim LibriSpeech test-clean NISQA TEST FOR NISQA TEST P501 NISQA TEST LIVETALK P23 EXP1 P23 EXP3 TCD VOIP NOIZEUS
Benchmarks: NISQA TEST FOR NISQA TEST P501 NISQA TEST LIVETALK P23 EXP1 P23 EXP3 TCD VOIP NOIZEUS TENCENT TENCENT-Rev VoiceMOS Test 1 VoiceMOS Test 2
Metrics: Pearson correlation Spearman correlation RMSE normalised mutual information
Baselines: L2 loss NOMAD NISQA NR-PESQ NR-SI SDR NORESQA-MOS Offline triplet model
Builds on: NOMAD Rank-and-Contrast (RnC) wav2vec 2.0 batch-all triplet loss (person re-identification) NISQA
Closest papers by meaning
- Audio Large Language Models Can Be Descriptive Speech Quality Evaluators (ICLR 2025 · similarity 0.87)
- ARECHO: Autoregressive Evaluation via Chain-Based Hypothesis Optimization for Speech Multi-Metric Estimation (NeurIPS 2025 · similarity 0.86)
- Towards Building a Language-Independent Speech Scoring Assessment (AAAI 2024 · similarity 0.85)
- Teaching Large Language Models to Regress Accurate Image Quality Scores Using Score Distribution (CVPR 2025 · similarity 0.84)
- Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech (ICLR 2024 · similarity 0.84)
- Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.