Research map / Large language models / Language, safety and interpretability
TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems
Christoph Minixhofer, Ondrej Klejch, Peter Bell
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TTSDS2 instead of original TTSDS | TTSDS metric · English average over 4 domains | Spearman correlation with subjective scores: 0.67 average, surpassing the original by 10% relative | Improved |
| use multilingual feature extractors (mHuBERT-147, XLSR-53) in TTSDS2 | TTSDS2 · Uriel+ typological distance | Spearman correlation: -0.39 -> -0.51 | Improved |
| replace WER features with ASR final-layer activations for Intelligibility | TTSDS · real-speech scores across domains | score robustness | Improved |
| replace HuBERT token-length prosody features with speaking rate | TTSDS · real-speech scores | score on real speech | Improved |
| remove ESC-50 from noise distractor datasets (ablation) | TTSDS2 · preliminary experiments | score consistency | Improved |
| use Speaker factor alone vs other factors (ablation) | TTSDS2 factors · Clean | Pearson correlation with MOS: Speaker 0.84 vs Generic 0.42, Prosody 0.38, Intelligibility 0.47 | Mixed |
| use Speaker factor alone vs other factors (ablation) | TTSDS2 factors · Noisy | Pearson correlation with MOS: Speaker 0.86 vs Generic 0.59, Prosody 0.46, Intelligibility 0.59 | Mixed |
| use Generic factor alone vs Speaker (ablation) | TTSDS2 factors · Kids | Pearson correlation with MOS: Generic 0.80 vs Speaker 0.70 | Mixed |
| use Speaker factor alone (ablation) | TTSDS2 factors · Wild | Pearson correlation with MOS: Speaker 0.59 vs Generic 0.53, Intelligibility 0.58, Prosody 0.34 | Mixed |
| compare TTSDS2 with X-Vector speaker similarity on Kids MOS | TTSDS2 · Kids MOS | Spearman correlation: 0.61 vs X-Vector 0.70 | No clear effect |
| compare TTSDS2 with UTMOS on Wild MOS | TTSDS2 · Wild MOS | Spearman correlation: 0.75 vs UTMOS -0.12 | Improved |
Closest papers by meaning
- TTA-Bench: A Comprehensive Benchmark for Evaluating Text-to-Audio Models (AAAI 2026 · similarity 0.89)
- Audio Large Language Models Can Be Descriptive Speech Quality Evaluators (ICLR 2025 · similarity 0.87)
- EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, Expressiveness, and Linguistic Challenges Using Model-as-a-Judge (NeurIPS 2025 · similarity 0.87)
- BIGOS V2 Benchmark for Polish ASR: Curated Datasets and Tools for Reproducible Evaluation (NeurIPS 2024 · similarity 0.86)
- Accelerating Unbiased LLM Evaluation via Synthetic Feedback (ICML 2025 · similarity 0.86)
- IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
